Trace number 3262838

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.19 1202.26

General information on the benchmark

NameSAT03/handmade/anton/UNSAT/
SGI_30_70_27_50_3-dir.shuffled-as.sat03-169.cnf
MD5SUMb5a637600c242725f15a247a84bdea1b
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables810
Number of clauses68270
Sum of the clauses size137296
Maximum clause length30
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 268243
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 527

Solver Data 

0.00/0.05	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262838-1304094215.cnf 
0.00/0.05	c Using SATTIMEOUT of 4800
0.00/0.05	c Using SATRAM of 15500
0.00/0.05	c Size of subset node = 32
0.00/0.05	c Variables = 810
0.00/0.05	c Clauses = 68270
0.00/0.05	c Literals = 137296
0.00/0.05	c Reduced size:
0.00/0.05	c Variables = 810
0.00/0.05	c Clauses = 68270
0.00/0.05	c Literals = 137296
0.00/0.05	c a2
0.05/0.09	c a1
0.09/0.19	c Independent sub-problems = 1
0.09/0.19	c Sub-problem with 810 variables and 68270 clauses
0.09/0.19	c using permutation method 4
0.09/0.19	c count = 1
0.09/0.19	c Ordering finished 0.170 seconds
0.09/0.19	c mapped 0.180 seconds
0.09/0.19	c permuted 0.180 seconds
0.09/0.19	c about to sort
0.09/0.19	c sorted
0.28/0.36	c clauses now 68270
0.28/0.36	c init finished 0.200 seconds
0.28/0.36	c tree built 0.350 seconds
0.28/0.36	c 
0.28/0.36	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.28/0.36	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.28/0.36	c ................................ | ..........                    |                                  0.3         0.0             
0.28/0.36	c ................................ | ..........                  1 |                                  0.3         2.9             
0.28/0.36	c ................................ | ..........                  2 |                                  0.3         5.7             
0.28/0.36	c ................................ | ..........                  3 |                                  0.3         8.6             
0.28/0.36	c ................................ | ..........                  4 |                                  0.3        11.4             
0.28/0.36	c ................................ | ..........                  5 |                                  0.3        14.3             
0.28/0.36	c ................................ | ..........               1000 |              14                  0.3      2857.1          346
95.96/24.31	c ................................ | ..........            5590000 | 151538       485850             96.0     58229.2      9602251
191.83/48.34	c ................................ | ..........            9627000 | 288449       865766            192.0     50140.6     14848849
287.68/72.38	c ................................ | ..........           12537000 | 402132       1204455           288.0     43526.7     20238645
383.93/96.42	c ................................ | ..........           15053000 | 505793       1484514           384.0     39198.5     25053067
479.80/120.46	c ................................ | *.........           17409000 | 603448       1718030           480.0     36265.7     28552337
575.63/144.49	c ................................ | *.........           21492000 | 699739       2009875           576.0     37311.2     31319491
671.87/168.52	c ................................ | *.........           24363000 | 784779       2271264           672.0     36254.5     35390901
767.73/192.57	c ................................ | *.........           27606000 | 853419       2535388           768.0     35945.3     39429699
863.97/216.61	c ................................ | *.........           30308000 | 912744       2759694           864.0     35077.9     42119453
959.82/240.64	c ................................ | **........           33002000 | 974955       2989261           960.0     34377.1     45660437
1055.71/264.68	c ................................ | **........           35285000 | 1024959      3184382          1056.0     33413.5     48331012
1151.96/288.72	c ................................ | **........           37253000 | 1078341      3368037          1152.0     32336.6     49296725
1247.79/312.76	c ................................ | **........           39422000 | 1130040      3531252          1248.0     31587.4     50717241
1344.06/336.81	c ................................ | **........           41260000 | 1176888      3706098          1344.1     30696.7     54406763
1439.91/360.82	c ................................ | ***.......           42684000 | 1215374      3871357          1440.0     29641.7     56131648
1535.77/384.88	c ................................ | ***.......           44429000 | 1257499      4040532          1536.1     28923.6     60813332
1632.03/408.90	c ................................ | ***.......           45826000 | 1296216      4217087          1632.0     28079.1     64625000
1727.90/432.93	c ................................ | ***.......           47566000 | 1332399      4452221          1728.0     27526.1     66765302
1823.75/456.96	c ................................ | ***.......           49526000 | 1371646      4656882          1824.0     27152.4     68508866
1919.98/481.02	c ................................ | ****......           51179000 | 1412984      4813419          1920.1     26654.8     69311360
2015.84/505.06	c ................................ | ****......           52234000 | 1459581      4992031          2016.1     25908.4     70151975
2111.69/529.08	c ................................ | ****......           53519000 | 1501611      5158340          2112.0     25340.2     70883335
2207.95/553.12	c ................................ | ****......           54414000 | 1547587      5275282          2208.1     24643.4     72212613
2303.81/577.16	c ................................ | ****......           55675000 | 1596051      5446832          2304.1     24163.9     75355143
2399.65/601.19	c ................................ | *****.....           57578000 | 1645524      5617770          2400.0     23990.8     77938385
2495.88/625.23	c ................................ | *****.....           59473000 | 1688523      5783528          2496.0     23827.1     78794172
2591.75/649.26	c ................................ | *****.....           61052000 | 1734434      5918973          2592.0     23553.9     80003143
2687.99/673.30	c ................................ | *****.....           62774000 | 1779941      6059876          2688.0     23353.3     82263126
2783.87/697.33	c ................................ | *****.....           64219000 | 1840582      6187960          2784.0     23067.1     85966609
2879.71/721.38	c ................................ | ******....           66329000 | 1920408      6349919          2880.1     23030.5     87921327
2975.98/745.41	c ................................ | ******....           67834000 | 1994866      6494996          2976.0     22793.5     89666354
3071.86/769.45	c ................................ | ******....           69115000 | 2063813      6620095          3072.0     22498.1     91242320
3167.69/793.48	c ................................ | ******....           70766000 | 2137920      6763069          3168.0     22337.6     93320079
3263.95/817.51	c ................................ | ******....           72442000 | 2190963      6880568          3264.0     22194.2     95014253
3359.81/841.55	c ................................ | *******...           73841000 | 2238010      7005050          3360.0     21976.4     96851878
3455.67/865.59	c ................................ | *******...           74747000 | 2277544      7103909          3456.0     21628.0     96889501
3551.93/889.66	c ................................ | *******...           75922000 | 2319463      7252191          3552.2     21373.5     96889501
3647.82/913.65	c ................................ | *******...           76772000 | 2362134      7379616          3648.0     21044.8     98366364
3743.64/937.69	c ................................ | *******...           78175000 | 2409906      7518758          3744.0     20880.0     98708755
3839.90/961.72	c ................................ | ********..           79624000 | 2459033      7678689          3840.0     20735.4    100218843
3935.74/985.76	c ................................ | ********..           80959000 | 2504463      7818558          3936.0     20568.9    101080744
4032.00/1009.80	c ................................ | ********..           82141000 | 2557233      7946308          4032.0     20372.2    101176026
4127.85/1033.83	c ................................ | ********..           83245000 | 2591222      8072335          4128.0     20165.9    103092266
4223.71/1057.89	c ................................ | ********..           84234000 | 2627692      8182086          4224.1     19941.4    104198193
4319.99/1081.91	c ................................ | *********.           85370000 | 2669164      8295424          4320.0     19761.5    105492522
4415.84/1105.95	c ................................ | *********.           86408000 | 2709999      8398891          4416.0     19566.9    106015782
4511.70/1129.98	c ................................ | *********.           87497000 | 2753896      8497908          4512.0     19392.0    106445383
4607.95/1154.02	c ................................ | *********.           88814000 | 2797281      8595639          4608.1     19273.6    108694069
4703.75/1178.07	c ................................ | *********.           90039000 | 2842105      8691309          4704.1     19140.7    109020337
4800.03/1202.10	c CPU limit (4800) reached
4800.03/1202.10	c Time = 4800.03 seconds
4800.03/1202.10	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3262838-1304094215/watcher-3262838-1304094215 -o /tmp/evaluation-result-3262838-1304094215/solver-3262838-1304094215 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262838-1304094215.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.02 8.08 8.00 2/181 10350
/proc/meminfo: memFree=24983016/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=9244 CPUtime=0 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) R 10348 10350 9884 0 -1 4202496 381 0 0 0 0 0 0 0 25 0 1 0 521019697 9465856 323 33554432000 4194304 4338100 140733814809504 18446744073709551615 246516495045 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10350/statm: 2311 324 147 36 0 216 0

[startup+0.0642699 s]
/proc/loadavg: 8.02 8.08 8.00 2/181 10350
/proc/meminfo: memFree=24983016/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=22688 CPUtime=0.05 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) R 10348 10350 9884 0 -1 4202496 3712 0 0 0 5 0 0 0 25 0 1 0 521019697 23232512 3653 33554432000 4194304 4338100 140733814809504 18446744073709551615 246516229336 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10350/statm: 5672 3653 153 36 0 3577 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 22688

[startup+0.100252 s]
/proc/loadavg: 8.02 8.08 8.00 2/181 10350
/proc/meminfo: memFree=24983016/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=23224 CPUtime=0.09 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) R 10348 10350 9884 0 -1 4202496 3785 0 0 0 9 0 0 0 25 0 1 0 521019697 23781376 3726 33554432000 4194304 4338100 140733814809504 18446744073709551615 4228108 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10350/statm: 5806 3726 153 36 0 3711 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 23224

[startup+0.300219 s]
/proc/loadavg: 8.02 8.08 8.00 2/181 10350
/proc/meminfo: memFree=24983016/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=43116 CPUtime=0.28 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) R 10348 10350 9884 0 -1 4202496 10630 0 0 0 27 1 0 0 25 0 1 0 521019697 44150784 8798 33554432000 4194304 4338100 140733814809504 18446744073709551615 4278146 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10350/statm: 10779 8798 164 36 0 8684 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 43116

[startup+1.18402 s]
/proc/loadavg: 8.02 8.08 8.00 2/181 10350
/proc/meminfo: memFree=24983016/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=255432 CPUtime=3.63 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 19135 0 0 0 359 4 0 0 25 0 5 0 521019697 261562368 17302 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 63858 17302 176 36 0 61763 0
Current children cumulated CPU time (s) 3.63
Current children cumulated vsize (KiB) 255432

[startup+1.50096 s]
/proc/loadavg: 8.02 8.08 8.00 2/181 10350
/proc/meminfo: memFree=24983016/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=261688 CPUtime=4.89 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 21039 0 0 0 485 4 0 0 25 0 5 0 521019697 267968512 19206 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 65422 19206 176 36 0 63327 0
Current children cumulated CPU time (s) 4.89
Current children cumulated vsize (KiB) 261688

[startup+3.10067 s]
/proc/loadavg: 7.54 7.97 7.97 9/190 10359
/proc/meminfo: memFree=24843520/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=290632 CPUtime=11.28 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 28600 0 0 0 1119 9 0 0 25 0 5 0 521019697 297607168 26767 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 72658 26768 176 36 0 70563 0
[pid=10350/tid=10352] ppid=10348 vsize=290632 CPUtime=2.71 cores=1,3,5,7
/proc/10350/task/10352/stat : 10352 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 5142 0 0 0 271 0 0 0 23 0 5 0 521019733 297607168 26768 33554432000 4194304 4338100 140733814809504 18446744073709551615 4253954 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10350/tid=10353] ppid=10348 vsize=290632 CPUtime=2.73 cores=1,3,5,7
/proc/10350/task/10353/stat : 10353 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 2790 0 0 0 270 3 0 0 23 0 5 0 521019733 297607168 26768 33554432000 4194304 4338100 140733814809504 18446744073709551615 246516229154 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10350/tid=10354] ppid=10348 vsize=290632 CPUtime=2.73 cores=1,3,5,7
/proc/10350/task/10354/stat : 10354 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 4874 0 0 0 273 0 0 0 25 0 5 0 521019733 297607168 26768 33554432000 4194304 4338100 140733814809504 18446744073709551615 246516229350 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10350/tid=10355] ppid=10348 vsize=290632 CPUtime=2.72 cores=1,3,5,7
/proc/10350/task/10355/stat : 10355 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 2459 0 0 0 270 2 0 0 23 0 5 0 521019733 297607168 26768 33554432000 4194304 4338100 140733814809504 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 11.28
Current children cumulated vsize (KiB) 290632

[startup+6.30109 s]
/proc/loadavg: 7.73 8.01 7.98 9/190 10359
/proc/meminfo: memFree=24592000/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=344996 CPUtime=24.06 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 41395 0 0 0 2384 22 0 0 25 0 5 0 521019697 353275904 39562 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 86249 39562 176 36 0 84154 0
[pid=10350/tid=10352] ppid=10348 vsize=344996 CPUtime=5.9 cores=1,3,5,7
/proc/10350/task/10352/stat : 10352 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 9897 0 0 0 588 2 0 0 25 0 5 0 521019733 353275904 39562 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254329 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10350/tid=10353] ppid=10348 vsize=344996 CPUtime=5.92 cores=1,3,5,7
/proc/10350/task/10353/stat : 10353 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 4912 0 0 0 583 9 0 0 25 0 5 0 521019733 353275904 39562 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10350/tid=10354] ppid=10348 vsize=344996 CPUtime=5.93 cores=1,3,5,7
/proc/10350/task/10354/stat : 10354 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 8156 0 0 0 593 0 0 0 25 0 5 0 521019733 353275904 39563 33554432000 4194304 4338100 140733814809504 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10350/tid=10355] ppid=10348 vsize=344996 CPUtime=5.93 cores=1,3,5,7
/proc/10350/task/10355/stat : 10355 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 5095 0 0 0 586 7 0 0 25 0 5 0 521019733 353275904 39563 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 24.06
Current children cumulated vsize (KiB) 344996

[startup+12.7069 s]
/proc/loadavg: 7.76 8.01 7.98 9/190 10359
/proc/meminfo: memFree=24238016/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=422132 CPUtime=49.65 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 61966 0 0 0 4914 51 0 0 25 0 5 0 521019697 432263168 60133 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 105533 60133 176 36 0 103438 0
[pid=10350/tid=10352] ppid=10348 vsize=422132 CPUtime=12.27 cores=1,3,5,7
/proc/10350/task/10352/stat : 10352 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 17451 0 0 0 1222 5 0 0 25 0 5 0 521019733 432263168 60133 33554432000 4194304 4338100 140733814809504 18446744073709551615 246516229185 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10350/tid=10353] ppid=10348 vsize=422132 CPUtime=12.32 cores=1,3,5,7
/proc/10350/task/10353/stat : 10353 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 8271 0 0 0 1210 22 0 0 25 0 5 0 521019733 432263168 60133 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10350/tid=10354] ppid=10348 vsize=422132 CPUtime=12.33 cores=1,3,5,7
/proc/10350/task/10354/stat : 10354 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 14387 0 0 0 1232 1 0 0 25 0 5 0 521019733 432263168 60133 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10350/tid=10355] ppid=10348 vsize=422132 CPUtime=12.33 cores=1,3,5,7
/proc/10350/task/10355/stat : 10355 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 8521 0 0 0 1214 19 0 0 25 0 5 0 521019733 432263168 60133 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 49.65
Current children cumulated vsize (KiB) 422132

[startup+25.5006 s]
/proc/loadavg: 7.79 8.01 7.98 9/190 10360
/proc/meminfo: memFree=23767688/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=558088 CPUtime=100.75 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 96000 0 0 0 9951 124 0 0 25 0 5 0 521019697 571482112 94167 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 139522 94167 176 36 0 137427 0
[pid=10350/tid=10352] ppid=10348 vsize=558088 CPUtime=24.99 cores=1,3,5,7
/proc/10350/task/10352/stat : 10352 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 33652 0 0 0 2487 12 0 0 25 0 5 0 521019733 571482112 94167 33554432000 4194304 4338100 140733814809504 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10350/tid=10353] ppid=10348 vsize=558088 CPUtime=25.12 cores=1,3,5,7
/proc/10350/task/10353/stat : 10353 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 12912 0 0 0 2458 54 0 0 25 0 5 0 521019733 571482112 94167 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10350/tid=10354] ppid=10348 vsize=558088 CPUtime=25.13 cores=1,3,5,7
/proc/10350/task/10354/stat : 10354 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 22398 0 0 0 2509 4 0 0 25 0 5 0 521019733 571482112 94167 33554432000 4194304 4338100 140733814809504 18446744073709551615 4253954 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10350/tid=10355] ppid=10348 vsize=558088 CPUtime=25.13 cores=1,3,5,7
/proc/10350/task/10355/stat : 10355 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 13702 0 0 0 2462 51 0 0 25 0 5 0 521019733 571482112 94167 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 100.75
Current children cumulated vsize (KiB) 558088

[startup+51.1009 s]
/proc/loadavg: 7.87 8.00 7.98 9/190 10361
/proc/meminfo: memFree=23100108/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=752668 CPUtime=203.03 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 144085 0 0 0 20027 276 0 0 25 0 5 0 521019697 770732032 142252 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 188167 142252 176 36 0 186072 0
[pid=10350/tid=10352] ppid=10348 vsize=752668 CPUtime=50.46 cores=1,3,5,7
/proc/10350/task/10352/stat : 10352 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 57544 0 0 0 5021 25 0 0 25 0 5 0 521019733 770732032 142252 33554432000 4194304 4338100 140733814809504 18446744073709551615 4253954 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10350/tid=10353] ppid=10348 vsize=752668 CPUtime=50.72 cores=1,3,5,7
/proc/10350/task/10353/stat : 10353 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 17926 0 0 0 4949 123 0 0 25 0 5 0 521019733 770732032 142252 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10350/tid=10354] ppid=10348 vsize=752668 CPUtime=50.73 cores=1,3,5,7
/proc/10350/task/10354/stat : 10354 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 35620 0 0 0 5065 8 0 0 25 0 5 0 521019733 770732032 142252 33554432000 4194304 4338100 140733814809504 18446744073709551615 4239484 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10350/tid=10355] ppid=10348 vsize=752668 CPUtime=50.72 cores=1,3,5,7
/proc/10350/task/10355/stat : 10355 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 19659 0 0 0 4956 116 0 0 25 0 5 0 521019733 770732032 142252 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 203.03
Current children cumulated vsize (KiB) 752668

[startup+102.301 s]
/proc/loadavg: 7.94 8.00 7.98 9/190 10362
/proc/meminfo: memFree=22139952/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=1091852 CPUtime=407.5 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 231380 0 0 0 40179 571 0 0 25 0 5 0 521019697 1118056448 229547 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 272963 229547 176 36 0 270868 0
[pid=10350/tid=10352] ppid=10348 vsize=1091852 CPUtime=101.37 cores=1,3,5,7
/proc/10350/task/10352/stat : 10352 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 91924 0 0 0 10095 42 0 0 25 0 5 0 521019733 1118056448 229547 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10350/tid=10353] ppid=10348 vsize=1091852 CPUtime=101.91 cores=1,3,5,7
/proc/10350/task/10353/stat : 10353 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 31485 0 0 0 9932 259 0 0 25 0 5 0 521019733 1118056448 229547 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10350/tid=10354] ppid=10348 vsize=1091852 CPUtime=101.92 cores=1,3,5,7
/proc/10350/task/10354/stat : 10354 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 59962 0 0 0 10177 15 0 0 25 0 5 0 521019733 1118056448 229547 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10350/tid=10355] ppid=10348 vsize=1091852 CPUtime=101.91 cores=1,3,5,7
/proc/10350/task/10355/stat : 10355 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 34673 0 0 0 9940 251 0 0 25 0 5 0 521019733 1118056448 229547 33554432000 4194304 4338100 140733814809504 18446744073709551615 4285747 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 407.5
Current children cumulated vsize (KiB) 1091852

[startup+162.307 s]
/proc/loadavg: 8.19 8.07 8.01 9/190 10363
/proc/meminfo: memFree=21391588/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=1420084 CPUtime=647.13 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 297816 0 0 0 63844 869 0 0 25 0 5 0 521019697 1454166016 295983 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 355021 295983 176 36 0 352926 0
[pid=10350/tid=10352] ppid=10348 vsize=1420084 CPUtime=161.03 cores=1,3,5,7
/proc/10350/task/10352/stat : 10352 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 131405 0 0 0 16041 62 0 0 25 0 5 0 521019733 1454166016 295983 33554432000 4194304 4338100 140733814809504 18446744073709551615 246516229363 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10350/tid=10353] ppid=10348 vsize=1420084 CPUtime=161.89 cores=1,3,5,7
/proc/10350/task/10353/stat : 10353 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 35277 0 0 0 15794 395 0 0 25 0 5 0 521019733 1454166016 295983 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10350/tid=10354] ppid=10348 vsize=1420084 CPUtime=161.91 cores=1,3,5,7
/proc/10350/task/10354/stat : 10354 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 78606 0 0 0 16170 21 0 0 25 0 5 0 521019733 1454166016 295984 33554432000 4194304 4338100 140733814809504 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10350/tid=10355] ppid=10348 vsize=1420084 CPUtime=161.9 cores=1,3,5,7
/proc/10350/task/10355/stat : 10355 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 39193 0 0 0 15803 387 0 0 25 0 5 0 521019733 1454166016 295984 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 647.13
Current children cumulated vsize (KiB) 1420084

[startup+222.301 s]
/proc/loadavg: 8.20 8.09 8.01 9/190 10365
/proc/meminfo: memFree=20621504/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=1692788 CPUtime=886.73 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 370262 0 0 0 87469 1204 0 0 25 0 5 0 521019697 1733414912 368429 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 423197 368429 176 36 0 421102 0
[pid=10350/tid=10352] ppid=10348 vsize=1692788 CPUtime=220.68 cores=1,3,5,7
/proc/10350/task/10352/stat : 10352 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 173989 0 0 0 21983 85 0 0 25 0 5 0 521019733 1733414912 368429 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0

################
# More data... #
################

/proc/meminfo: memFree=17191788/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=3264928 CPUtime=3043.48 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 775140 0 0 0 300593 3755 0 0 25 0 5 0 521019697 3343286272 773307 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 816232 773307 176 36 0 814137 0
[pid=10350/tid=10352] ppid=10348 vsize=3264928 CPUtime=759.09 cores=1,3,5,7
/proc/10350/task/10352/stat : 10352 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 351610 0 0 0 75712 197 0 0 25 0 5 0 521019733 3343286272 773307 33554432000 4194304 4338100 140733814809504 18446744073709551615 4253950 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10350/tid=10353] ppid=10348 vsize=3264928 CPUtime=761.5 cores=1,3,5,7
/proc/10350/task/10353/stat : 10353 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 83193 0 0 0 74402 1748 0 0 25 0 5 0 521019733 3343286272 773307 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10350/tid=10354] ppid=10348 vsize=3264928 CPUtime=761.86 cores=1,3,5,7
/proc/10350/task/10354/stat : 10354 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 233465 0 0 0 76118 68 0 0 25 0 5 0 521019733 3343286272 773307 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10350/tid=10355] ppid=10348 vsize=3264928 CPUtime=760.63 cores=1,3,5,7
/proc/10350/task/10355/stat : 10355 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 93536 0 0 0 74326 1737 0 0 25 0 5 0 521019733 3343286272 773307 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3043.48
Current children cumulated vsize (KiB) 3264928

[startup+822.307 s]
/proc/loadavg: 8.12 8.10 8.03 9/190 10381
/proc/meminfo: memFree=16923416/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=3410400 CPUtime=3283.14 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 813029 0 0 0 324362 3952 0 0 25 0 5 0 521019697 3492249600 811196 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 852600 811196 176 36 0 850505 0
[pid=10350/tid=10352] ppid=10348 vsize=3410400 CPUtime=819.09 cores=1,3,5,7
/proc/10350/task/10352/stat : 10352 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 377964 0 0 0 81700 209 0 0 25 0 5 0 521019733 3492249600 811196 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528646862 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10350/tid=10353] ppid=10348 vsize=3410400 CPUtime=821.49 cores=1,3,5,7
/proc/10350/task/10353/stat : 10353 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 84744 0 0 0 80310 1839 0 0 25 0 5 0 521019733 3492249600 811196 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10350/tid=10354] ppid=10348 vsize=3410400 CPUtime=821.86 cores=1,3,5,7
/proc/10350/task/10354/stat : 10354 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 242919 0 0 0 82114 72 0 0 25 0 5 0 521019733 3492249600 811196 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10350/tid=10355] ppid=10348 vsize=3410400 CPUtime=820.31 cores=1,3,5,7
/proc/10350/task/10355/stat : 10355 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 94066 0 0 0 80203 1828 0 0 25 0 5 0 521019733 3492249600 811196 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3283.14
Current children cumulated vsize (KiB) 3410400

[startup+882.301 s]
/proc/loadavg: 8.13 8.11 8.03 9/190 10383
/proc/meminfo: memFree=16753484/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=3456040 CPUtime=3522.77 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 826312 0 0 0 348143 4134 0 0 25 0 5 0 521019697 3538984960 824479 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 864010 824479 176 36 0 861915 0
[pid=10350/tid=10352] ppid=10348 vsize=3456040 CPUtime=879.09 cores=1,3,5,7
/proc/10350/task/10352/stat : 10352 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 386154 0 0 0 87693 216 0 0 25 0 5 0 521019733 3538984960 824479 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10350/tid=10353] ppid=10348 vsize=3456040 CPUtime=881.38 cores=1,3,5,7
/proc/10350/task/10353/stat : 10353 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 85301 0 0 0 86214 1924 0 0 25 0 5 0 521019733 3538984960 824479 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10350/tid=10354] ppid=10348 vsize=3456040 CPUtime=881.84 cores=1,3,5,7
/proc/10350/task/10354/stat : 10354 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 246987 0 0 0 88110 74 0 0 25 0 5 0 521019733 3538984960 824479 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10350/tid=10355] ppid=10348 vsize=3456040 CPUtime=880.07 cores=1,3,5,7
/proc/10350/task/10355/stat : 10355 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 94534 0 0 0 86091 1916 0 0 25 0 5 0 521019733 3538984960 824479 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254462 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3522.77
Current children cumulated vsize (KiB) 3456040

[startup+942.301 s]
/proc/loadavg: 8.16 8.12 8.04 9/190 10384
/proc/meminfo: memFree=16559256/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=3524564 CPUtime=3762.41 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 842946 0 0 0 371892 4349 0 0 25 0 5 0 521019697 3609153536 841113 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 881141 841113 176 36 0 879046 0
[pid=10350/tid=10352] ppid=10348 vsize=3524564 CPUtime=939.08 cores=1,3,5,7
/proc/10350/task/10352/stat : 10352 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 394515 0 0 0 93684 224 0 0 25 0 5 0 521019733 3609153536 841113 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10350/tid=10353] ppid=10348 vsize=3524564 CPUtime=941.2 cores=1,3,5,7
/proc/10350/task/10353/stat : 10353 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 85909 0 0 0 92093 2027 0 0 25 0 5 0 521019733 3609153536 841113 33554432000 4194304 4338100 140733814809504 18446744073709551615 4240537 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10350/tid=10354] ppid=10348 vsize=3524564 CPUtime=941.84 cores=1,3,5,7
/proc/10350/task/10354/stat : 10354 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 254051 0 0 0 94107 77 0 0 25 0 5 0 521019733 3609153536 841113 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10350/tid=10355] ppid=10348 vsize=3524564 CPUtime=939.9 cores=1,3,5,7
/proc/10350/task/10355/stat : 10355 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 95135 0 0 0 91973 2017 0 0 25 0 5 0 521019733 3609153536 841113 33554432000 4194304 4338100 140733814809504 18446744073709551615 4253950 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3762.41
Current children cumulated vsize (KiB) 3524564

[startup+1002.3 s]
/proc/loadavg: 8.19 8.13 8.04 9/190 10386
/proc/meminfo: memFree=16360196/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=3601352 CPUtime=4002.03 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 863465 0 0 0 395649 4554 0 0 25 0 5 0 521019697 3687784448 861632 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 900338 861632 176 36 0 898243 0
[pid=10350/tid=10352] ppid=10348 vsize=3601352 CPUtime=999.07 cores=1,3,5,7
/proc/10350/task/10352/stat : 10352 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 406835 0 0 0 99673 234 0 0 25 0 5 0 521019733 3687784448 861632 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10350/tid=10353] ppid=10348 vsize=3601352 CPUtime=1001.18 cores=1,3,5,7
/proc/10350/task/10353/stat : 10353 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 86726 0 0 0 97995 2123 0 0 25 0 5 0 521019733 3687784448 861632 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254291 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10350/tid=10354] ppid=10348 vsize=3601352 CPUtime=1001.83 cores=1,3,5,7
/proc/10350/task/10354/stat : 10354 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 260832 0 0 0 100103 80 0 0 25 0 5 0 521019733 3687784448 861632 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10350/tid=10355] ppid=10348 vsize=3601352 CPUtime=999.56 cores=1,3,5,7
/proc/10350/task/10355/stat : 10355 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 95736 0 0 0 97843 2113 0 0 25 0 5 0 521019733 3687784448 861632 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254285 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4002.03
Current children cumulated vsize (KiB) 3601352

[startup+1062.3 s]
/proc/loadavg: 8.26 8.16 8.06 9/190 10387
/proc/meminfo: memFree=16044568/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=3715016 CPUtime=4241.69 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 893288 0 0 0 419450 4719 0 0 25 0 5 0 521019697 3804176384 891455 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 928754 891455 176 36 0 926659 0
[pid=10350/tid=10352] ppid=10348 vsize=3715016 CPUtime=1059.07 cores=1,3,5,7
/proc/10350/task/10352/stat : 10352 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 422908 0 0 0 105666 241 0 0 25 0 5 0 521019733 3804176384 891455 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10350/tid=10353] ppid=10348 vsize=3715016 CPUtime=1061.18 cores=1,3,5,7
/proc/10350/task/10353/stat : 10353 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 87717 0 0 0 103916 2202 0 0 25 0 5 0 521019733 3804176384 891455 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10350/tid=10354] ppid=10348 vsize=3715016 CPUtime=1061.83 cores=1,3,5,7
/proc/10350/task/10354/stat : 10354 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 273062 0 0 0 106098 85 0 0 25 0 5 0 521019733 3804176384 891455 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10350/tid=10355] ppid=10348 vsize=3715016 CPUtime=1059.22 cores=1,3,5,7
/proc/10350/task/10355/stat : 10355 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 96265 0 0 0 103734 2188 0 0 25 0 5 0 521019733 3804176384 891455 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4241.69
Current children cumulated vsize (KiB) 3715016

[startup+1122.3 s]
/proc/loadavg: 8.23 8.16 8.06 9/190 10389
/proc/meminfo: memFree=15858636/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=3766384 CPUtime=4481.34 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 907570 0 0 0 443249 4885 0 0 25 0 5 0 521019697 3856777216 905737 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 941596 905737 176 36 0 939501 0
[pid=10350/tid=10352] ppid=10348 vsize=3766384 CPUtime=1119.06 cores=1,3,5,7
/proc/10350/task/10352/stat : 10352 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 432769 0 0 0 111660 246 0 0 25 0 5 0 521019733 3856777216 905737 33554432000 4194304 4338100 140733814809504 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10350/tid=10353] ppid=10348 vsize=3766384 CPUtime=1121.14 cores=1,3,5,7
/proc/10350/task/10353/stat : 10353 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 88888 0 0 0 109830 2284 0 0 25 0 5 0 521019733 3856777216 905737 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10350/tid=10354] ppid=10348 vsize=3766384 CPUtime=1121.83 cores=1,3,5,7
/proc/10350/task/10354/stat : 10354 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 275372 0 0 0 112096 87 0 0 25 0 5 0 521019733 3856777216 905737 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10350/tid=10355] ppid=10348 vsize=3766384 CPUtime=1118.93 cores=1,3,5,7
/proc/10350/task/10355/stat : 10355 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 97205 0 0 0 109628 2265 0 0 25 0 5 0 521019733 3856777216 905737 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4481.34
Current children cumulated vsize (KiB) 3766384

[startup+1182.3 s]
/proc/loadavg: 8.25 8.18 8.07 10/190 10391
/proc/meminfo: memFree=15648776/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=3855288 CPUtime=4720.94 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 930198 0 0 0 467005 5089 0 0 25 0 5 0 521019697 3947814912 928365 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 963822 928365 176 36 0 961727 0
[pid=10350/tid=10352] ppid=10348 vsize=3855288 CPUtime=1179.04 cores=1,3,5,7
/proc/10350/task/10352/stat : 10352 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 444854 0 0 0 117651 253 0 0 25 0 5 0 521019733 3947814912 928365 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10350/tid=10353] ppid=10348 vsize=3855288 CPUtime=1180.91 cores=1,3,5,7
/proc/10350/task/10353/stat : 10353 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 89318 0 0 0 115715 2376 0 0 25 0 5 0 521019733 3947814912 928365 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10350/tid=10354] ppid=10348 vsize=3855288 CPUtime=1181.69 cores=1,3,5,7
/proc/10350/task/10354/stat : 10354 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 283846 0 0 0 118078 91 0 0 25 0 5 0 521019733 3947814912 928365 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10350/tid=10355] ppid=10348 vsize=3855288 CPUtime=1178.91 cores=1,3,5,7
/proc/10350/task/10355/stat : 10355 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 98844 0 0 0 115527 2364 0 0 25 0 5 0 521019733 3947814912 928365 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4720.94
Current children cumulated vsize (KiB) 3855288



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.1 s]
/proc/loadavg: 8.18 8.16 8.07 9/190 10391
/proc/meminfo: memFree=15594448/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=3883836 CPUtime=4800.03 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) S 10348 10350 9884 0 -1 4202496 938803 0 0 0 474833 5170 0 0 25 0 5 0 521019697 3977048064 936970 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 970959 936970 176 36 0 968864 0
[pid=10350/tid=10352] ppid=10348 vsize=3883836 CPUtime=1198.83 cores=1,3,5,7
/proc/10350/task/10352/stat : 10352 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 448592 0 0 0 119628 255 0 0 25 0 5 0 521019733 3977048064 936970 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10350/tid=10353] ppid=10348 vsize=3883836 CPUtime=1200.71 cores=1,3,5,7
/proc/10350/task/10353/stat : 10353 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 89520 0 0 0 117656 2415 0 0 25 0 5 0 521019733 3977048064 936970 33554432000 4194304 4338100 140733814809504 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10350/tid=10354] ppid=10348 vsize=3883836 CPUtime=1201.38 cores=1,3,5,7
/proc/10350/task/10354/stat : 10354 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 288093 0 0 0 120046 92 0 0 25 0 5 0 521019733 3977048064 936970 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10350/tid=10355] ppid=10348 vsize=3883836 CPUtime=1198.71 cores=1,3,5,7
/proc/10350/task/10355/stat : 10355 (ttsth-5-4) R 10348 10350 9884 0 -1 4202560 99262 0 0 0 117467 2404 0 0 25 0 5 0 521019733 3977048064 936970 33554432000 4194304 4338100 140733814809504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.03
Current children cumulated vsize (KiB) 3883836

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202.21 s]
/proc/loadavg: 8.18 8.16 8.07 9/190 10391
/proc/meminfo: memFree=15594448/32951124 swapFree=67111352/67111528
[pid=10350] ppid=10348 vsize=0 CPUtime=4800.14 cores=1,3,5,7
/proc/10350/stat : 10350 (ttsth-5-4) Z 10348 10350 9884 0 -1 4203548 938804 0 0 0 474833 5181 0 0 18 0 2 0 521019697 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10350/statm: 0 0 0 0 0 0 0
[pid=10350/tid=10354] ppid=10348 vsize=0 CPUtime=1201.48 cores=1,3,5,7
/proc/10350/task/10354/stat : 10354 (ttsth-5-4) R 10348 10350 9884 0 -1 4203588 288093 0 0 0 120046 102 0 0 25 0 2 0 521019733 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4800.14
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1202.26
CPU time (s): 4800.19
CPU user time (s): 4748.33
CPU system time (s): 51.8591
CPU usage (%): 399.265
Max. virtual memory (cumulated for all children) (KiB): 3883836

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4748.33
system time used= 51.8591
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 938804
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 7978
involuntary context switches= 23862

runsolver used 1.9677 second user time and 4.5983 second system time

The end

Launcher Data 

Begin job on node134 at 2011-04-29 18:23:35
IDJOB=3262838
IDBENCH=83896
IDSOLVER=1654
FILE ID=node134/3262838-1304094215
RUNJOBID= node134-1304093010-10232
PBS_JOBID= 13170571
Free space on /tmp= 71584 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT03/handmade/anton/UNSAT/SGI_30_70_27_50_3-dir.shuffled-as.sat03-169.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3262838-1304094215/watcher-3262838-1304094215 -o /tmp/evaluation-result-3262838-1304094215/solver-3262838-1304094215 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262838-1304094215.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= b5a637600c242725f15a247a84bdea1b
RANDOM SEED=2077176773

node134.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 7186.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      24983172 kB
Buffers:       1490104 kB
Cached:        5857052 kB
SwapCached:         64 kB
Active:        1499848 kB
Inactive:      5946764 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      24983172 kB
SwapTotal:    67111528 kB
SwapFree:     67111352 kB
Dirty:            2796 kB
Writeback:           0 kB
AnonPages:       99460 kB
Mapped:          15200 kB
Slab:           459372 kB
PageTables:       4524 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   276096 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71580 MiB
End job on node134 at 2011-04-29 18:43:39