Trace number 3263190

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.25 1202.25

General information on the benchmark

NameSAT03/handmade/anton/UNSAT/
SGI_30_70_26_60_1-dir.shuffled-as.sat03-167.cnf
MD5SUMb970e9358cd0cd1a9cfd3d4afb2e830d
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables780
Number of clauses65612
Sum of the clauses size131952
Maximum clause length30
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 265586
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 526

Solver Data 

0.00/0.04	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3263190-1304095775.cnf 
0.00/0.04	c Using SATTIMEOUT of 4800
0.00/0.04	c Using SATRAM of 15500
0.00/0.04	c Size of subset node = 32
0.00/0.04	c Variables = 780
0.00/0.04	c Clauses = 65612
0.00/0.04	c Literals = 131952
0.00/0.04	c Reduced size:
0.00/0.04	c Variables = 780
0.00/0.04	c Clauses = 65612
0.00/0.04	c Literals = 131952
0.00/0.04	c a2
0.06/0.08	c a1
0.09/0.18	c Independent sub-problems = 1
0.09/0.18	c Sub-problem with 780 variables and 65612 clauses
0.09/0.18	c using permutation method 4
0.09/0.18	c count = 1
0.09/0.18	c Ordering finished 0.160 seconds
0.09/0.18	c mapped 0.170 seconds
0.09/0.18	c permuted 0.170 seconds
0.09/0.18	c about to sort
0.09/0.18	c sorted
0.29/0.34	c clauses now 65612
0.29/0.34	c init finished 0.190 seconds
0.29/0.34	c tree built 0.330 seconds
0.29/0.34	c 
0.29/0.34	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.29/0.34	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.29/0.34	c ................................ | ..........                    |                                  0.3         0.0             
0.29/0.34	c ................................ | ..........                  1 |                                  0.3         3.0             
0.29/0.34	c ................................ | ..........                  2 |                                  0.3         6.1             
0.29/0.34	c ................................ | ..........                  3 |                                  0.3         9.1             
0.29/0.34	c ................................ | ..........                  4 |                                  0.3        12.1             
0.29/0.34	c ................................ | ..........                  5 |                                  0.3        15.2             
0.29/0.34	c ................................ | ..........               1000 |              19                  0.3      3030.3         1022
95.65/24.29	c ................................ | ..........            5397000 | 107336       474204             96.0     56218.8      9154050
191.90/48.33	c ................................ | ..........            9172000 | 208740       817006            192.0     47763.4     14970276
287.76/72.36	c ................................ | ..........           11770000 | 301869       1035775           288.0     40863.8     19139126
383.62/96.39	c ................................ | ..........           14817000 | 390979       1275961           384.0     38585.9     22234297
479.87/120.43	c ................................ | *.........           17665000 | 483400       1487194           480.0     36801.3     24767964
575.73/144.47	c ................................ | *.........           20525000 | 571028       1726682           576.0     35632.4     26990221
671.99/168.51	c ................................ | *.........           22173000 | 639262       1863793           672.0     32993.1     31033776
767.88/192.54	c ................................ | *.........           23851000 | 703170       1955888           768.0     31054.8     35972930
863.69/216.58	c ................................ | *.........           25673000 | 765784       2125855           864.0     29714.1     41172556
959.93/240.62	c ................................ | **........           27684000 | 828603       2264952           960.0     28836.6     41789333
1055.79/264.66	c ................................ | **........           29617000 | 905927       2400672          1056.0     28045.6     44072574
1151.62/288.69	c ................................ | **........           31324000 | 981236       2528881          1152.0     27191.0     45532999
1247.87/312.74	c ................................ | **........           33054000 | 1043972      2669390          1248.0     26484.9     46388424
1343.74/336.76	c ................................ | **........           34463000 | 1092478      2798925          1344.0     25642.1     47784732
1439.61/360.80	c ................................ | ***.......           36047000 | 1145060      2907685          1440.0     25032.6     50337636
1535.87/384.84	c ................................ | ***.......           37497000 | 1207973      3070924          1536.0     24411.6     51921024
1632.11/408.90	c ................................ | ***.......           38936000 | 1263402      3213312          1632.1     23855.9     53775319
1727.95/432.94	c ................................ | ***.......           40123000 | 1321597      3332790          1728.1     23217.7     55052257
1823.82/456.95	c ................................ | ***.......           41285000 | 1369659      3431635          1824.0     22633.7     55461681
1920.06/481.00	c ................................ | ****......           42733000 | 1428885      3525260          1920.1     22256.1     56275203
2015.89/505.03	c ................................ | ****......           44203000 | 1481115      3629491          2016.0     21926.0     57231525
2111.78/529.07	c ................................ | ****......           45964000 | 1542988      3751851          2112.1     21762.6     57541752
2208.00/553.10	c ................................ | ****......           47459000 | 1597220      3865868          2208.0     21494.0     57865114
2303.86/577.14	c ................................ | ****......           48808000 | 1649109      3999745          2304.0     21183.8     57865114
2399.73/601.18	c ................................ | *****.....           50184000 | 1694825      4116679          2400.1     20909.4     57951182
2495.98/625.22	c ................................ | *****.....           51415000 | 1740787      4229628          2496.1     20598.3     58740197
2591.83/649.25	c ................................ | *****.....           52333000 | 1784312      4312438          2592.1     20189.8     59922132
2687.73/673.29	c ................................ | *****.....           53374000 | 1828129      4381083          2688.1     19856.0     61292825
2783.96/697.33	c ................................ | *****.....           54616000 | 1873887      4465678          2784.1     19617.3     61538102
2879.82/721.35	c ................................ | ******....           56211000 | 1928910      4584143          2880.0     19517.6     63143207
2975.67/745.38	c ................................ | ******....           57915000 | 1989242      4686628          2976.0     19460.7     64934165
3071.93/769.44	c ................................ | ******....           59807000 | 2041488      4805667          3072.1     19467.7     66064071
3167.79/793.48	c ................................ | ******....           60967000 | 2084386      4908250          3168.1     19244.0     66252940
3263.65/817.49	c ................................ | ******....           62281000 | 2131531      5028703          3264.0     19081.2     66252940
3359.95/841.53	c ................................ | *******...           63495000 | 2184637      5136550          3360.1     18897.0     67003101
3455.79/865.55	c ................................ | *******...           64469000 | 2227684      5235671          3456.0     18654.1     68199601
3551.64/889.60	c ................................ | *******...           65386000 | 2272422      5315299          3552.0     18408.0     69300119
3647.81/913.65	c ................................ | *******...           66575000 | 2314618      5398979          3648.0     18249.7     70657045
3743.67/937.69	c ................................ | *******...           67862000 | 2360598      5491910          3744.1     18125.2     72253739
3839.90/961.73	c ................................ | ********..           69276000 | 2419394      5579128          3840.0     18040.5     73372370
3935.76/985.77	c ................................ | ********..           70699000 | 2470275      5678285          3936.1     17961.9     74349464
4032.02/1009.81	c ................................ | ********..           72142000 | 2525862      5756761          4032.1     17892.1     76019708
4127.84/1033.84	c ................................ | ********..           73953000 | 2584318      5879363          4128.0     17914.9     77775584
4223.68/1057.88	c ................................ | ********..           75142000 | 2637250      5978135          4224.0     17789.3     79505864
4319.95/1081.92	c ................................ | *********.           76043000 | 2680459      6060001          4320.0     17602.4     80820196
4415.77/1105.97	c ................................ | *********.           77365000 | 2732499      6154010          4416.1     17519.0     80959258
4512.00/1130.03	c ................................ | *********.           78325000 | 2774337      6243649          4512.1     17358.7     81519886
4607.86/1154.07	c ................................ | *********.           79467000 | 2826853      6332256          4608.1     17245.0     81519886
4703.72/1178.08	c ................................ | *********.           80395000 | 2867375      6425738          4704.1     17090.6     81519886
4799.95/1202.14	c CPU limit (4800) reached
4799.95/1202.14	c Time = 4800.13 seconds
4799.95/1202.14	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3263190-1304095775/watcher-3263190-1304095775 -o /tmp/evaluation-result-3263190-1304095775/solver-3263190-1304095775 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3263190-1304095775.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.57 7.77 7.92 2/174 3089
/proc/meminfo: memFree=27649024/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=9408 CPUtime=0 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) R 3087 3089 2524 0 -1 4202496 431 0 0 0 0 0 0 0 21 0 1 0 521175188 9633792 373 33554432000 4194304 4338100 140734196430608 18446744073709551615 218905391813 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/3089/statm: 2352 374 147 36 0 257 0

[startup+0.077983 s]
/proc/loadavg: 6.57 7.77 7.92 2/174 3089
/proc/meminfo: memFree=27649024/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=22152 CPUtime=0.06 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) R 3087 3089 2524 0 -1 4202496 3580 0 0 0 6 0 0 0 21 0 1 0 521175188 22683648 3521 33554432000 4194304 4338100 140734196430608 18446744073709551615 4206987 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/3089/statm: 5538 3521 153 36 0 3443 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 22152

[startup+0.100982 s]
/proc/loadavg: 6.57 7.77 7.92 2/174 3089
/proc/meminfo: memFree=27649024/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=18344 CPUtime=0.09 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) R 3087 3089 2524 0 -1 4202496 3857 0 0 0 9 0 0 0 21 0 1 0 521175188 18784256 2385 33554432000 4194304 4338100 140734196430608 18446744073709551615 4296613 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/3089/statm: 4586 2386 155 36 0 2491 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 18344

[startup+0.300945 s]
/proc/loadavg: 6.57 7.77 7.92 2/174 3089
/proc/meminfo: memFree=27649024/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=44864 CPUtime=0.29 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) R 3087 3089 2524 0 -1 4202496 10823 0 0 0 28 1 0 0 22 0 1 0 521175188 45940736 9222 33554432000 4194304 4338100 140734196430608 18446744073709551615 218905391813 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/3089/statm: 11216 9227 164 36 0 9121 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 44864

[startup+0.700836 s]
/proc/loadavg: 6.57 7.77 7.92 2/174 3089
/proc/meminfo: memFree=27649024/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=240840 CPUtime=1.76 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 15398 0 0 0 174 2 0 0 22 0 5 0 521175188 246620160 13796 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 60210 13796 176 36 0 58115 0
Current children cumulated CPU time (s) 1.76
Current children cumulated vsize (KiB) 240840

[startup+1.50069 s]
/proc/loadavg: 6.57 7.77 7.92 6/182 3101
/proc/meminfo: memFree=27549448/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=257008 CPUtime=4.96 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 19809 0 0 0 490 6 0 0 22 0 5 0 521175188 263176192 18207 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 64252 18207 176 36 0 62157 0
[pid=3089/tid=3097] ppid=3087 vsize=257008 CPUtime=1.15 cores=1,3,5,7
/proc/3089/task/3097/stat : 3097 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 1924 0 0 0 115 0 0 0 19 0 5 0 521175222 263176192 18207 33554432000 4194304 4338100 140734196430608 18446744073709551615 218905126322 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=3089/tid=3098] ppid=3087 vsize=257008 CPUtime=1.14 cores=1,3,5,7
/proc/3089/task/3098/stat : 3098 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 925 0 0 0 113 1 0 0 19 0 5 0 521175222 263176192 18207 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254347 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=3089/tid=3099] ppid=3087 vsize=257008 CPUtime=1.15 cores=1,3,5,7
/proc/3089/task/3099/stat : 3099 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 2351 0 0 0 115 0 0 0 25 0 5 0 521175222 263176192 18207 33554432000 4194304 4338100 140734196430608 18446744073709551615 218905125904 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=257008 CPUtime=1.14 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 1917 0 0 0 113 1 0 0 19 0 5 0 521175222 263176192 18208 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254239 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4.96
Current children cumulated vsize (KiB) 257008

[startup+3.1004 s]
/proc/loadavg: 6.57 7.77 7.92 6/182 3101
/proc/meminfo: memFree=27550812/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=285980 CPUtime=11.35 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 26937 0 0 0 1123 12 0 0 22 0 5 0 521175188 292843520 25335 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 71495 25335 176 36 0 69400 0
[pid=3089/tid=3097] ppid=3087 vsize=285980 CPUtime=2.75 cores=1,3,5,7
/proc/3089/task/3097/stat : 3097 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 4116 0 0 0 274 1 0 0 23 0 5 0 521175222 292843520 25335 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254275 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=3089/tid=3098] ppid=3087 vsize=285980 CPUtime=2.74 cores=1,3,5,7
/proc/3089/task/3098/stat : 3098 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 2044 0 0 0 270 4 0 0 23 0 5 0 521175222 292843520 25335 33554432000 4194304 4338100 140734196430608 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=3089/tid=3099] ppid=3087 vsize=285980 CPUtime=2.74 cores=1,3,5,7
/proc/3089/task/3099/stat : 3099 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 4530 0 0 0 274 0 0 0 25 0 5 0 521175222 292843520 25335 33554432000 4194304 4338100 140734196430608 18446744073709551615 4253966 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=285980 CPUtime=2.74 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 3555 0 0 0 271 3 0 0 23 0 5 0 521175222 292843520 25335 33554432000 4194304 4338100 140734196430608 18446744073709551615 218905427499 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.35
Current children cumulated vsize (KiB) 285980

[startup+6.30082 s]
/proc/loadavg: 6.68 7.77 7.92 9/186 3106
/proc/meminfo: memFree=27348932/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=330828 CPUtime=24.13 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 38790 0 0 0 2386 27 0 0 22 0 5 0 521175188 338767872 37188 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 82707 37188 176 36 0 80612 0
[pid=3089/tid=3097] ppid=3087 vsize=330828 CPUtime=5.95 cores=1,3,5,7
/proc/3089/task/3097/stat : 3097 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 7823 0 0 0 592 3 0 0 25 0 5 0 521175222 338767872 37188 33554432000 4194304 4338100 140734196430608 18446744073709551615 218905125904 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=3089/tid=3098] ppid=3087 vsize=330828 CPUtime=5.94 cores=1,3,5,7
/proc/3089/task/3098/stat : 3098 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 4092 0 0 0 583 11 0 0 25 0 5 0 521175222 338767872 37188 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=3089/tid=3099] ppid=3087 vsize=330828 CPUtime=5.95 cores=1,3,5,7
/proc/3089/task/3099/stat : 3099 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 7960 0 0 0 594 1 0 0 25 0 5 0 521175222 338767872 37188 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917543718 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=330828 CPUtime=5.92 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 6223 0 0 0 583 9 0 0 25 0 5 0 521175222 338767872 37188 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.13
Current children cumulated vsize (KiB) 330828

[startup+12.7007 s]
/proc/loadavg: 6.79 7.78 7.92 9/186 3106
/proc/meminfo: memFree=27037040/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=426572 CPUtime=49.7 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 62490 0 0 0 4907 63 0 0 22 0 5 0 521175188 436809728 60888 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 106643 60888 176 36 0 104548 0
[pid=3089/tid=3097] ppid=3087 vsize=426572 CPUtime=12.35 cores=1,3,5,7
/proc/3089/task/3097/stat : 3097 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 15769 0 0 0 1229 6 0 0 25 0 5 0 521175222 436809728 60888 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254352 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=3089/tid=3098] ppid=3087 vsize=426572 CPUtime=12.34 cores=1,3,5,7
/proc/3089/task/3098/stat : 3098 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 8220 0 0 0 1207 27 0 0 25 0 5 0 521175222 436809728 60888 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=3089/tid=3099] ppid=3087 vsize=426572 CPUtime=12.34 cores=1,3,5,7
/proc/3089/task/3099/stat : 3099 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 14986 0 0 0 1232 2 0 0 25 0 5 0 521175222 436809728 60888 33554432000 4194304 4338100 140734196430608 18446744073709551615 4239484 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=426572 CPUtime=12.28 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 10823 0 0 0 1205 23 0 0 25 0 5 0 521175222 436809728 60888 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 49.7
Current children cumulated vsize (KiB) 426572

[startup+25.5003 s]
/proc/loadavg: 7.05 7.79 7.92 9/186 3106
/proc/meminfo: memFree=26616940/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=540128 CPUtime=100.82 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 91045 0 0 0 9958 124 0 0 22 0 5 0 521175188 553091072 89093 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 135032 89094 176 36 0 132937 0
[pid=3089/tid=3097] ppid=3087 vsize=540128 CPUtime=25.15 cores=1,3,5,7
/proc/3089/task/3097/stat : 3097 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 27113 0 0 0 2503 12 0 0 25 0 5 0 521175222 553091072 89094 33554432000 4194304 4338100 140734196430608 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=3089/tid=3098] ppid=3087 vsize=540128 CPUtime=25.14 cores=1,3,5,7
/proc/3089/task/3098/stat : 3098 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 12494 0 0 0 2458 56 0 0 25 0 5 0 521175222 553091072 89094 33554432000 4194304 4338100 140734196430608 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=3089/tid=3099] ppid=3087 vsize=540128 CPUtime=25.14 cores=1,3,5,7
/proc/3089/task/3099/stat : 3099 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 23459 0 0 0 2510 4 0 0 25 0 5 0 521175222 553091072 89094 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=540128 CPUtime=25.01 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 15288 0 0 0 2453 48 0 0 25 0 5 0 521175222 553091072 89094 33554432000 4194304 4338100 140734196430608 18446744073709551615 218905126118 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 100.82
Current children cumulated vsize (KiB) 540128

[startup+51.1007 s]
/proc/loadavg: 7.53 7.84 7.93 10/186 3107
/proc/meminfo: memFree=25981372/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=737364 CPUtime=203.08 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 141769 0 0 0 20073 235 0 0 22 0 5 0 521175188 755060736 139817 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 184341 139817 176 36 0 182246 0
[pid=3089/tid=3097] ppid=3087 vsize=737364 CPUtime=50.75 cores=1,3,5,7
/proc/3089/task/3097/stat : 3097 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 45767 0 0 0 5054 21 0 0 25 0 5 0 521175222 755060736 139817 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=3089/tid=3098] ppid=3087 vsize=737364 CPUtime=50.75 cores=1,3,5,7
/proc/3089/task/3098/stat : 3098 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 17821 0 0 0 4969 106 0 0 25 0 5 0 521175222 755060736 139817 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254321 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=3089/tid=3099] ppid=3087 vsize=737364 CPUtime=50.74 cores=1,3,5,7
/proc/3089/task/3099/stat : 3099 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 42393 0 0 0 5065 9 0 0 25 0 5 0 521175222 755060736 139817 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254275 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=737364 CPUtime=50.47 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 23096 0 0 0 4952 95 0 0 25 0 5 0 521175222 755060736 139817 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.08
Current children cumulated vsize (KiB) 737364

[startup+102.306 s]
/proc/loadavg: 7.87 7.88 7.94 9/186 3108
/proc/meminfo: memFree=25097112/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=1000804 CPUtime=407.6 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 208017 0 0 0 40323 437 0 0 22 0 5 0 521175188 1024823296 206065 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 250201 206065 176 36 0 248106 0
[pid=3089/tid=3097] ppid=3087 vsize=1000804 CPUtime=101.94 cores=1,3,5,7
/proc/3089/task/3097/stat : 3097 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 82614 0 0 0 10152 42 0 0 25 0 5 0 521175222 1024823296 206066 33554432000 4194304 4338100 140734196430608 18446744073709551615 218905126264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=3089/tid=3098] ppid=3087 vsize=1000804 CPUtime=101.95 cores=1,3,5,7
/proc/3089/task/3098/stat : 3098 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 23246 0 0 0 10003 192 0 0 25 0 5 0 521175222 1024823296 206066 33554432000 4194304 4338100 140734196430608 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=3089/tid=3099] ppid=3087 vsize=1000804 CPUtime=101.93 cores=1,3,5,7
/proc/3089/task/3099/stat : 3099 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 61599 0 0 0 10179 14 0 0 25 0 5 0 521175222 1024823296 206066 33554432000 4194304 4338100 140734196430608 18446744073709551615 4256770 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=1000804 CPUtime=101.41 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 27867 0 0 0 9956 185 0 0 25 0 5 0 521175222 1024823296 206066 33554432000 4194304 4338100 140734196430608 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 407.6
Current children cumulated vsize (KiB) 1000804

[startup+162.301 s]
/proc/loadavg: 8.10 7.93 7.95 9/186 3110
/proc/meminfo: memFree=24266656/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=1217716 CPUtime=647.23 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 262976 0 0 0 64099 624 0 0 22 0 5 0 521175188 1246941184 261024 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 304429 261024 176 36 0 302334 0
[pid=3089/tid=3097] ppid=3087 vsize=1217716 CPUtime=161.61 cores=1,3,5,7
/proc/3089/task/3097/stat : 3097 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 113186 0 0 0 16100 61 0 0 25 0 5 0 521175222 1246941184 261024 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=3089/tid=3098] ppid=3087 vsize=1217716 CPUtime=161.93 cores=1,3,5,7
/proc/3089/task/3098/stat : 3098 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 26578 0 0 0 15918 275 0 0 25 0 5 0 521175222 1246941184 261024 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917556917 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=3089/tid=3099] ppid=3087 vsize=1217716 CPUtime=161.92 cores=1,3,5,7
/proc/3089/task/3099/stat : 3099 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 78594 0 0 0 16173 19 0 0 25 0 5 0 521175222 1246941184 261024 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=1217716 CPUtime=161.39 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 31926 0 0 0 15874 265 0 0 25 0 5 0 521175222 1246941184 261024 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917556917 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 647.23
Current children cumulated vsize (KiB) 1217716


################
# More data... #
################

/proc/meminfo: memFree=20520072/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=2395672 CPUtime=2803.93 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 542128 0 0 0 278336 2057 0 0 22 0 5 0 521175188 2453168128 540176 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 598918 540176 176 36 0 596823 0
[pid=3089/tid=3097] ppid=3087 vsize=2395672 CPUtime=699.08 cores=1,3,5,7
/proc/3089/task/3097/stat : 3097 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 244388 0 0 0 69758 150 0 0 25 0 5 0 521175222 2453168128 540176 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=3089/tid=3098] ppid=3087 vsize=2395672 CPUtime=701.89 cores=1,3,5,7
/proc/3089/task/3098/stat : 3098 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 47082 0 0 0 69258 931 0 0 25 0 5 0 521175222 2453168128 540176 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=3089/tid=3099] ppid=3087 vsize=2395672 CPUtime=701.26 cores=1,3,5,7
/proc/3089/task/3099/stat : 3099 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 184336 0 0 0 70074 52 0 0 25 0 5 0 521175222 2453168128 540176 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=2395672 CPUtime=701.32 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 53630 0 0 0 69212 920 0 0 25 0 5 0 521175222 2453168128 540176 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2803.93
Current children cumulated vsize (KiB) 2395672

[startup+762.307 s]
/proc/loadavg: 8.06 8.05 8.00 9/186 3140
/proc/meminfo: memFree=20246736/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=2520172 CPUtime=3043.59 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 574873 0 0 0 302151 2208 0 0 22 0 5 0 521175188 2580656128 572921 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 630043 572921 176 36 0 627948 0
[pid=3089/tid=3097] ppid=3087 vsize=2520172 CPUtime=758.77 cores=1,3,5,7
/proc/3089/task/3097/stat : 3097 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 267908 0 0 0 75712 165 0 0 25 0 5 0 521175222 2580656128 572921 33554432000 4194304 4338100 140734196430608 18446744073709551615 218905126650 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=3089/tid=3098] ppid=3087 vsize=2520172 CPUtime=761.9 cores=1,3,5,7
/proc/3089/task/3098/stat : 3098 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 47379 0 0 0 75192 998 0 0 25 0 5 0 521175222 2580656128 572921 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=3089/tid=3099] ppid=3087 vsize=2520172 CPUtime=761.23 cores=1,3,5,7
/proc/3089/task/3099/stat : 3099 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 192976 0 0 0 76069 54 0 0 25 0 5 0 521175222 2580656128 572921 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=2520172 CPUtime=761.32 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 53918 0 0 0 75144 988 0 0 25 0 5 0 521175222 2580656128 572921 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917556917 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3043.59
Current children cumulated vsize (KiB) 2520172

[startup+822.301 s]
/proc/loadavg: 8.09 8.06 8.00 9/186 3141
/proc/meminfo: memFree=20125800/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=2540788 CPUtime=3283.22 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 581046 0 0 0 325980 2342 0 0 22 0 5 0 521175188 2601766912 579094 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 635197 579094 176 36 0 633102 0
[pid=3089/tid=3097] ppid=3087 vsize=2540788 CPUtime=818.76 cores=1,3,5,7
/proc/3089/task/3097/stat : 3097 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 271780 0 0 0 81704 172 0 0 25 0 5 0 521175222 2601766912 579094 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=3089/tid=3098] ppid=3087 vsize=2540788 CPUtime=821.88 cores=1,3,5,7
/proc/3089/task/3098/stat : 3098 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 47688 0 0 0 81125 1063 0 0 25 0 5 0 521175222 2601766912 579094 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917556917 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=3089/tid=3099] ppid=3087 vsize=2540788 CPUtime=820.91 cores=1,3,5,7
/proc/3089/task/3099/stat : 3099 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 194663 0 0 0 82035 56 0 0 25 0 5 0 521175222 2601766912 579094 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=2540788 CPUtime=821.31 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 54223 0 0 0 81083 1048 0 0 25 0 5 0 521175222 2601766912 579094 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917556917 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3283.22
Current children cumulated vsize (KiB) 2540788

[startup+882.301 s]
/proc/loadavg: 8.18 8.09 8.02 9/186 3147
/proc/meminfo: memFree=19938048/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=2623964 CPUtime=3522.88 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 602914 0 0 0 349821 2467 0 0 22 0 5 0 521175188 2686939136 600962 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 655991 600962 176 36 0 653896 0
[pid=3089/tid=3097] ppid=3087 vsize=2623964 CPUtime=878.76 cores=1,3,5,7
/proc/3089/task/3097/stat : 3097 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 279717 0 0 0 87696 180 0 0 25 0 5 0 521175222 2686939136 600962 33554432000 4194304 4338100 140734196430608 18446744073709551615 218905125953 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=3089/tid=3098] ppid=3087 vsize=2623964 CPUtime=881.88 cores=1,3,5,7
/proc/3089/task/3098/stat : 3098 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 47922 0 0 0 87070 1118 0 0 25 0 5 0 521175222 2686939136 600962 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917556917 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=3089/tid=3099] ppid=3087 vsize=2623964 CPUtime=880.57 cores=1,3,5,7
/proc/3089/task/3099/stat : 3099 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 208127 0 0 0 87996 61 0 0 25 0 5 0 521175222 2686939136 600962 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=2623964 CPUtime=881.31 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 54456 0 0 0 87027 1104 0 0 25 0 5 0 521175222 2686939136 600962 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917556917 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3522.88
Current children cumulated vsize (KiB) 2623964

[startup+942.3 s]
/proc/loadavg: 7.96 8.07 8.01 9/186 3150
/proc/meminfo: memFree=19659500/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=2741812 CPUtime=3762.45 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 633181 0 0 0 373648 2597 0 0 22 0 5 0 521175188 2807615488 631229 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 685453 631229 176 36 0 683358 0
[pid=3089/tid=3097] ppid=3087 vsize=2741812 CPUtime=938.76 cores=1,3,5,7
/proc/3089/task/3097/stat : 3097 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 298458 0 0 0 93683 193 0 0 25 0 5 0 521175222 2807615488 631229 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=3089/tid=3098] ppid=3087 vsize=2741812 CPUtime=941.84 cores=1,3,5,7
/proc/3089/task/3098/stat : 3098 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 48109 0 0 0 93009 1175 0 0 25 0 5 0 521175222 2807615488 631229 33554432000 4194304 4338100 140734196430608 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=3089/tid=3099] ppid=3087 vsize=2741812 CPUtime=940.2 cores=1,3,5,7
/proc/3089/task/3099/stat : 3099 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 219263 0 0 0 93954 66 0 0 25 0 5 0 521175222 2807615488 631229 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=2741812 CPUtime=941.29 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 54659 0 0 0 92969 1160 0 0 25 0 5 0 521175222 2807615488 631229 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917556917 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3762.45
Current children cumulated vsize (KiB) 2741812

[startup+1002.3 s]
/proc/loadavg: 8.06 8.07 8.01 9/186 3152
/proc/meminfo: memFree=19465032/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=2834768 CPUtime=4002.03 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 657173 0 0 0 397497 2706 0 0 22 0 5 0 521175188 2902802432 655221 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 708692 655221 176 36 0 706597 0
[pid=3089/tid=3097] ppid=3087 vsize=2834768 CPUtime=998.71 cores=1,3,5,7
/proc/3089/task/3097/stat : 3097 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 315612 0 0 0 99667 204 0 0 25 0 5 0 521175222 2902802432 655221 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254418 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=3089/tid=3098] ppid=3087 vsize=2834768 CPUtime=1001.83 cores=1,3,5,7
/proc/3089/task/3098/stat : 3098 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 48368 0 0 0 98962 1221 0 0 25 0 5 0 521175222 2902802432 655221 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917556917 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=3089/tid=3099] ppid=3087 vsize=2834768 CPUtime=999.84 cores=1,3,5,7
/proc/3089/task/3099/stat : 3099 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 225585 0 0 0 99916 68 0 0 25 0 5 0 521175222 2902802432 655221 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254030 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=2834768 CPUtime=1001.28 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 54916 0 0 0 98919 1209 0 0 25 0 5 0 521175222 2902802432 655221 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917556917 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4002.03
Current children cumulated vsize (KiB) 2834768

[startup+1062.3 s]
/proc/loadavg: 8.02 8.05 8.00 9/186 3154
/proc/meminfo: memFree=19206576/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=2971120 CPUtime=4241.65 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 692452 0 0 0 421335 2830 0 0 22 0 5 0 521175188 3042426880 690500 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 742780 690500 176 36 0 740685 0
[pid=3089/tid=3097] ppid=3087 vsize=2971120 CPUtime=1058.35 cores=1,3,5,7
/proc/3089/task/3097/stat : 3097 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 336741 0 0 0 105618 217 0 0 25 0 5 0 521175222 3042426880 690500 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=3089/tid=3098] ppid=3087 vsize=2971120 CPUtime=1061.83 cores=1,3,5,7
/proc/3089/task/3098/stat : 3098 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 48708 0 0 0 104906 1277 0 0 25 0 5 0 521175222 3042426880 690500 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917556917 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=3089/tid=3099] ppid=3087 vsize=2971120 CPUtime=1059.81 cores=1,3,5,7
/proc/3089/task/3099/stat : 3099 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 237494 0 0 0 105908 73 0 0 25 0 5 0 521175222 3042426880 690500 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254291 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=2971120 CPUtime=1061.29 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 56817 0 0 0 104869 1260 0 0 25 0 5 0 521175222 3042426880 690500 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917556917 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4241.65
Current children cumulated vsize (KiB) 2971120

[startup+1122.3 s]
/proc/loadavg: 8.07 8.06 8.00 9/186 3155
/proc/meminfo: memFree=19009596/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=3029140 CPUtime=4481.29 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 707933 0 0 0 445193 2936 0 0 22 0 5 0 521175188 3101839360 705981 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 757285 705981 176 36 0 755190 0
[pid=3089/tid=3097] ppid=3087 vsize=3029140 CPUtime=1118 cores=1,3,5,7
/proc/3089/task/3097/stat : 3097 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 346792 0 0 0 111575 225 0 0 25 0 5 0 521175222 3101839360 705981 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=3089/tid=3098] ppid=3087 vsize=3029140 CPUtime=1121.83 cores=1,3,5,7
/proc/3089/task/3098/stat : 3098 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 48928 0 0 0 110856 1327 0 0 25 0 5 0 521175222 3101839360 705981 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917556917 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=3089/tid=3099] ppid=3087 vsize=3029140 CPUtime=1119.79 cores=1,3,5,7
/proc/3089/task/3099/stat : 3099 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 241178 0 0 0 111905 74 0 0 25 0 5 0 521175222 3101839360 705981 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254326 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=3029140 CPUtime=1121.29 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 58343 0 0 0 110823 1306 0 0 25 0 5 0 521175222 3101839360 705981 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917556917 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4481.29
Current children cumulated vsize (KiB) 3029140

[startup+1182.3 s]
/proc/loadavg: 8.15 8.08 8.01 9/186 3157
/proc/meminfo: memFree=18916160/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=3031780 CPUtime=4720.87 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) S 3087 3089 2524 0 -1 4202496 709503 0 0 0 469053 3034 0 0 22 0 5 0 521175188 3104542720 707551 33554432000 4194304 4338100 140734196430608 18446744073709551615 218917533669 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 757945 707551 176 36 0 755850 0
[pid=3089/tid=3097] ppid=3087 vsize=3031780 CPUtime=1177.75 cores=1,3,5,7
/proc/3089/task/3097/stat : 3097 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 347524 0 0 0 117545 230 0 0 25 0 5 0 521175222 3104542720 707551 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=3089/tid=3098] ppid=3087 vsize=3031780 CPUtime=1181.83 cores=1,3,5,7
/proc/3089/task/3098/stat : 3098 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 49161 0 0 0 116809 1374 0 0 25 0 5 0 521175222 3104542720 707551 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=3089/tid=3099] ppid=3087 vsize=3031780 CPUtime=1179.68 cores=1,3,5,7
/proc/3089/task/3099/stat : 3099 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 241546 0 0 0 117893 75 0 0 25 0 5 0 521175222 3104542720 707551 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=3031780 CPUtime=1181.26 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4202560 58580 0 0 0 116774 1352 0 0 25 0 5 0 521175222 3104542720 707551 33554432000 4194304 4338100 140734196430608 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4720.87
Current children cumulated vsize (KiB) 3031780



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.2 s]
/proc/loadavg: 8.11 8.07 8.01 6/183 3157
/proc/meminfo: memFree=20409744/32951124 swapFree=67111356/67111528
[pid=3089] ppid=3087 vsize=0 CPUtime=4800.19 cores=1,3,5,7
/proc/3089/stat : 3089 (ttsth-5-4) Z 3087 3089 2524 0 -1 4203548 709961 0 0 0 476947 3072 0 0 18 0 2 0 521175188 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/3089/statm: 0 0 0 0 0 0 0
[pid=3089/tid=3101] ppid=3087 vsize=0 CPUtime=1201.17 cores=1,3,5,7
/proc/3089/task/3101/stat : 3101 (ttsth-5-4) R 3087 3089 2524 0 -1 4203588 58692 0 0 0 118744 1373 0 0 25 0 2 0 521175222 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.19
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 0
Real time (s): 1202.25
CPU time (s): 4800.25
CPU user time (s): 4769.48
CPU system time (s): 30.7703
CPU usage (%): 399.271
Max. virtual memory (cumulated for all children) (KiB): 3032308

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4769.48
system time used= 30.7703
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 709961
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4807
involuntary context switches= 19434

runsolver used 1.9527 second user time and 4.6183 second system time

The end

Launcher Data 

Begin job on node137 at 2011-04-29 18:49:35
IDJOB=3263190
IDBENCH=83934
IDSOLVER=1654
FILE ID=node137/3263190-1304095775
RUNJOBID= node137-1304095775-3075
PBS_JOBID= 13170581
Free space on /tmp= 71620 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT03/handmade/anton/UNSAT/SGI_30_70_26_60_1-dir.shuffled-as.sat03-167.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3263190-1304095775/watcher-3263190-1304095775 -o /tmp/evaluation-result-3263190-1304095775/solver-3263190-1304095775 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3263190-1304095775.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= b970e9358cd0cd1a9cfd3d4afb2e830d
RANDOM SEED=378359717

node137.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      27649304 kB
Buffers:        919016 kB
Cached:        3850648 kB
SwapCached:         56 kB
Active:         336212 kB
Inactive:      4500308 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      27649304 kB
SwapTotal:    67111528 kB
SwapFree:     67111356 kB
Dirty:            2124 kB
Writeback:           0 kB
AnonPages:       66796 kB
Mapped:          14484 kB
Slab:           401836 kB
PageTables:       4312 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   188328 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71616 MiB
End job on node137 at 2011-04-29 19:09:39