Trace number 3249284

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.37 1203.09

General information on the benchmark

Namecrafted/mosoi/289/
289-unsat-5x41.cnf
MD5SUM420773b7b438dd18c8a88a15fb5bcae1
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables820
Number of clauses34235
Sum of the clauses size134480
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 21230
Number of clauses of size 30
Number of clauses of size 433005
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.03	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3249284-1304061417.cnf 
0.00/0.03	c Using SATTIMEOUT of 4800
0.00/0.03	c Using SATRAM of 15500
0.00/0.03	c Size of subset node = 32
0.00/0.03	c Variables = 820
0.00/0.03	c Clauses = 34235
0.00/0.03	c Literals = 134480
0.00/0.03	c Reduced size:
0.00/0.03	c Variables = 820
0.00/0.03	c Clauses = 34235
0.00/0.03	c Literals = 134480
0.00/0.03	c a2
0.09/0.10	c a1
0.19/0.20	c Independent sub-problems = 1
0.19/0.20	c Sub-problem with 820 variables and 34235 clauses
0.19/0.20	c using permutation method 4
0.19/0.20	c count = 1
0.19/0.20	c Ordering finished 0.190 seconds
0.19/0.20	c mapped 0.190 seconds
0.19/0.20	c permuted 0.190 seconds
0.19/0.20	c about to sort
0.19/0.20	c sorted
0.19/0.28	c clauses now 34235
0.19/0.28	c init finished 0.210 seconds
0.19/0.28	c tree built 0.270 seconds
0.19/0.28	c 
0.19/0.28	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.19/0.28	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.19/0.28	c ................................ | ..........                    |                                  0.3         0.0             
0.19/0.28	c ................................ | ..........                  1 |                                  0.3         3.7             
0.19/0.28	c ................................ | ..........                  2 |                                  0.3         7.4             
0.19/0.28	c ................................ | ..........                  3 |                                  0.3        11.1             
0.19/0.28	c ................................ | ..........                  4 |                                  0.3        14.8             
0.19/0.28	c ................................ | ..........                  5 |                                  0.3        18.5             
0.19/0.28	c ................................ | ..........               1000 | 16           429                 0.3      3571.4         1329
95.80/24.25	c ................................ | ..........            1452000 | 223570       1390422            96.0     15121.8     10053783
192.04/48.33	c ................................ | ..........            1838000 | 305885       1742126           192.2      9564.9     13342172
287.88/72.35	c ................................ | ..........            2103000 | 365532       1968079           288.1      7300.1     14942702
383.70/96.38	c ................................ | ..........            2348000 | 422638       2160929           384.0      6113.9     16689412
479.92/120.42	c ................................ | *.........            2669000 | 499016       2414538           480.0      5560.1     18801134
575.78/144.49	c ................................ | *.........            2887000 | 554166       2567007           576.1      5010.9     20211639
672.01/168.51	c ................................ | *.........            3080000 | 598087       2715500           672.1      4582.7     21784686
768.67/192.72	c ................................ | *.........            3187000 | 623428       2793819           768.8      4145.6     23168986
864.10/216.65	c ................................ | *.........            3287000 | 647471       2864128           864.3      3803.0     24209225
959.94/240.62	c ................................ | **........            3411000 | 676616       2958482           960.0      3553.0     25464561
1056.16/264.79	c ................................ | **........            3530000 | 704938       3046710          1056.5      3341.1     26188629
1151.95/288.71	c ................................ | **........            4176000 | 792121       3559360          1152.0      3625.0     30411686
1248.07/312.84	c ................................ | **........            5133000 | 933795       4445142          1248.2      4112.3     40959363
1343.90/336.83	c ................................ | **........            5707000 | 1054716      5045966          1344.0      4246.2     48167105
1439.93/360.93	c ................................ | ***.......            6377000 | 1190186      5768536          1440.1      4428.3     49107214
1535.65/384.99	c ................................ | ***.......            7026000 | 1343000      6290407          1536.0      4574.2     53781076
1631.80/409.07	c ................................ | ***.......            7729000 | 1515254      6923216          1632.1      4735.6     56860609
1727.87/433.14	c ................................ | ***.......            8449000 | 1670705      7686860          1728.1      4889.3     59377604
1824.03/457.21	c ................................ | ***.......            9106000 | 1810663      8222215          1824.1      4992.1     65677460
1919.77/481.25	c ................................ | ****......            9708000 | 1963502      8728017          1920.0      5056.2     71665846
2015.85/505.35	c ................................ | ****......           10330000 | 2115575      9298639          2016.0      5123.9     78035899
2111.89/529.45	c ................................ | ****......           10968000 | 2257439      9860184          2112.1      5193.0     82133192
2207.74/553.48	c ................................ | ****......           11403000 | 2355370      10202520         2208.1      5164.2     86275301
2304.02/577.66	c ................................ | ****......           11899000 | 2464487      10643745         2304.2      5163.9     91125514
2399.76/601.68	c ................................ | *****.....           12384000 | 2582054      10963213         2400.1      5159.8     98937818
2496.35/625.84	c ................................ | *****.....           12816000 | 2676966      11321234         2496.5      5133.6    104851793
2591.70/649.78	c ................................ | *****.....           13198000 | 2749725      11655724         2592.1      5091.7    110120950
2687.96/673.87	c ................................ | *****.....           13528000 | 2816441      11927660         2688.2      5032.3    113964415
2784.08/697.91	c ................................ | *****.....           13997000 | 2917746      12318641         2784.1      5027.4    117626703
2879.95/721.99	c ................................ | ******....           14261000 | 2973581      12545784         2880.3      4951.2    120674406
2976.18/746.00	c ................................ | ******....           14459000 | 3019805      12718239         2976.2      4858.2    122157336
3072.35/770.16	c ................................ | ******....           14725000 | 3082267      12959596         3072.6      4792.3    123175504
3168.10/794.12	c ................................ | ******....           14968000 | 3138142      13180752         3168.2      4724.5    124761833
3264.26/818.23	c ................................ | ******....           15303000 | 3216081      13497379         3264.4      4687.8    126588541
3360.03/842.21	c ................................ | *******...           15463000 | 3253079      13627299         3360.1      4601.9    126993004
3455.75/866.26	c ................................ | *******...           15689000 | 3305239      13818719         3456.0      4539.6    129750917
3552.06/890.39	c ................................ | *******...           15908000 | 3353787      14011337         3552.4      4478.1    133548891
3647.90/914.39	c ................................ | *******...           16119000 | 3398488      14197162         3648.3      4418.3    136418370
3743.70/938.39	c ................................ | *******...           16571000 | 3461783      14537585         3744.1      4425.9    138438072
3839.73/962.48	c ................................ | ********..           17137000 | 3542374      15007701         3840.1      4462.7    143571429
3935.86/986.59	c ................................ | ********..           17619000 | 3613216      15418200         3936.2      4476.1    145757928
4033.29/1010.97	c ................................ | ********..           17675000 | 3624326      15463690         4033.6      4381.9    146291938
4127.97/1034.65	c ................................ | ********..           17723000 | 3633979      15501473         4128.2      4293.2    146327701
4225.44/1059.03	c ................................ | ********..           17781000 | 3648772      15536727         4225.6      4208.0    146403123
4319.73/1082.69	c ................................ | *********.           17833000 | 3662146      15567652         4320.1      4127.9    146403123
4415.99/1106.72	c ................................ | *********.           17887000 | 3675614      15600242         4416.1      4050.4    146403123
4513.06/1131.08	c ................................ | *********.           17947000 | 3691767      15631658         4513.4      3976.4    146440416
4608.13/1154.81	c ................................ | *********.           18006000 | 3705035      15668174         4608.2      3907.4    146609403
4704.76/1179.03	c ................................ | *********.           18091000 | 3726329      15716043         4704.9      3845.1    146928340
4799.84/1202.88	c CPU limit (4800) reached
4799.84/1202.88	c Time = 4800.16 seconds
4799.84/1202.88	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3249284-1304061417/watcher-3249284-1304061417 -o /tmp/evaluation-result-3249284-1304061417/solver-3249284-1304061417 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3249284-1304061417.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.08 8.09 8.02 2/177 29971
/proc/meminfo: memFree=26583920/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=9408 CPUtime=0 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) R 29969 29971 29500 0 -1 4202496 406 0 0 0 0 0 0 0 25 0 1 0 498557203 9633792 348 33554432000 4194304 4338100 140734570303184 18446744073709551615 228095111877 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/29971/statm: 2352 349 147 36 0 257 0

[startup+0.0705639 s]
/proc/loadavg: 8.08 8.09 8.02 2/177 29971
/proc/meminfo: memFree=26583920/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=19688 CPUtime=0.06 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) R 29969 29971 29500 0 -1 4202496 2998 0 0 0 6 0 0 0 25 0 1 0 498557203 20160512 2939 33554432000 4194304 4338100 140734570303184 18446744073709551615 228095147539 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/29971/statm: 4922 2939 153 36 0 2827 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 19688

[startup+0.100554 s]
/proc/loadavg: 8.08 8.09 8.02 2/177 29971
/proc/meminfo: memFree=26583920/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=19688 CPUtime=0.09 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) R 29969 29971 29500 0 -1 4202496 2998 0 0 0 9 0 0 0 25 0 1 0 498557203 20160512 2939 33554432000 4194304 4338100 140734570303184 18446744073709551615 4206987 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/29971/statm: 4922 2939 153 36 0 2827 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 19688

[startup+0.301049 s]
/proc/loadavg: 8.08 8.09 8.02 2/177 29971
/proc/meminfo: memFree=26583920/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=220424 CPUtime=0.46 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 8923 0 0 0 45 1 0 0 25 0 5 0 498557203 225714176 7402 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 55205 7709 176 36 0 53110 0
Current children cumulated CPU time (s) 0.46
Current children cumulated vsize (KiB) 220424

[startup+1.19832 s]
/proc/loadavg: 8.08 8.09 8.02 2/177 29971
/proc/meminfo: memFree=26583920/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=227604 CPUtime=3.92 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 11792 0 0 0 390 2 0 0 25 0 5 0 498557203 233066496 10271 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 56901 10271 176 36 0 54806 0
Current children cumulated CPU time (s) 3.92
Current children cumulated vsize (KiB) 227604

[startup+2.19813 s]
/proc/loadavg: 8.08 8.09 8.02 2/177 29971
/proc/meminfo: memFree=26583920/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=306444 CPUtime=7.92 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 15892 0 0 0 788 4 0 0 25 0 5 0 498557203 313798656 14371 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 76611 14372 176 36 0 74516 0
Current children cumulated CPU time (s) 7.92
Current children cumulated vsize (KiB) 306444

[startup+3.10097 s]
/proc/loadavg: 7.60 7.98 7.99 9/186 29980
/proc/meminfo: memFree=26419728/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=315960 CPUtime=11.53 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 18508 0 0 0 1146 7 0 0 25 0 5 0 498557203 323543040 16987 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 78990 16987 176 36 0 76895 0
[pid=29971/tid=29973] ppid=29969 vsize=315960 CPUtime=2.8 cores=1,3,5,7
/proc/29971/task/29973/stat : 29973 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 1459 0 0 0 278 2 0 0 20 0 5 0 498557231 323543040 16987 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29971/tid=29974] ppid=29969 vsize=315960 CPUtime=2.81 cores=1,3,5,7
/proc/29971/task/29974/stat : 29974 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 2934 0 0 0 280 1 0 0 21 0 5 0 498557231 323543040 16987 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=29971/tid=29975] ppid=29969 vsize=315960 CPUtime=2.81 cores=1,3,5,7
/proc/29971/task/29975/stat : 29975 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 1815 0 0 0 280 1 0 0 21 0 5 0 498557231 323543040 16987 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254362 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29971/tid=29976] ppid=29969 vsize=315960 CPUtime=2.8 cores=1,3,5,7
/proc/29971/task/29976/stat : 29976 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 4072 0 0 0 279 1 0 0 24 0 5 0 498557231 323543040 16987 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254433 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 11.53
Current children cumulated vsize (KiB) 315960

[startup+6.30037 s]
/proc/loadavg: 7.63 7.98 7.99 7/186 29980
/proc/meminfo: memFree=26243480/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=367064 CPUtime=24.29 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 30866 0 0 0 2417 12 0 0 25 0 5 0 498557203 375873536 29345 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 91766 29345 176 36 0 89671 0
[pid=29971/tid=29973] ppid=29969 vsize=367064 CPUtime=5.99 cores=1,3,5,7
/proc/29971/task/29973/stat : 29973 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 3286 0 0 0 596 3 0 0 19 0 5 0 498557231 375873536 29345 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=29971/tid=29974] ppid=29969 vsize=367064 CPUtime=6.01 cores=1,3,5,7
/proc/29971/task/29974/stat : 29974 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 6923 0 0 0 598 3 0 0 19 0 5 0 498557231 375873536 29345 33554432000 4194304 4338100 140734570303184 18446744073709551615 4256087 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29971/tid=29975] ppid=29969 vsize=367064 CPUtime=6 cores=1,3,5,7
/proc/29971/task/29975/stat : 29975 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 3815 0 0 0 598 2 0 0 25 0 5 0 498557231 375873536 29345 33554432000 4194304 4338100 140734570303184 18446744073709551615 228094845986 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29971/tid=29976] ppid=29969 vsize=367064 CPUtime=5.99 cores=1,3,5,7
/proc/29971/task/29976/stat : 29976 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 8614 0 0 0 597 2 0 0 25 0 5 0 498557231 375873536 29345 33554432000 4194304 4338100 140734570303184 18446744073709551615 4240537 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.29
Current children cumulated vsize (KiB) 367064

[startup+12.7002 s]
/proc/loadavg: 7.66 7.98 7.99 9/186 29980
/proc/meminfo: memFree=25964488/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=456544 CPUtime=49.85 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 53516 0 0 0 4958 27 0 0 25 0 5 0 498557203 467501056 51995 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 114136 51995 176 36 0 112041 0
[pid=29971/tid=29973] ppid=29969 vsize=456544 CPUtime=12.38 cores=1,3,5,7
/proc/29971/task/29973/stat : 29973 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 7312 0 0 0 1232 6 0 0 25 0 5 0 498557231 467501056 51995 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=29971/tid=29974] ppid=29969 vsize=456544 CPUtime=12.4 cores=1,3,5,7
/proc/29971/task/29974/stat : 29974 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 12498 0 0 0 1233 7 0 0 25 0 5 0 498557231 467501056 51995 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254384 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29971/tid=29975] ppid=29969 vsize=456544 CPUtime=12.4 cores=1,3,5,7
/proc/29971/task/29975/stat : 29975 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 8820 0 0 0 1235 5 0 0 25 0 5 0 498557231 467501056 51995 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29971/tid=29976] ppid=29969 vsize=456544 CPUtime=12.35 cores=1,3,5,7
/proc/29971/task/29976/stat : 29976 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 16658 0 0 0 1229 6 0 0 25 0 5 0 498557231 467501056 51995 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254326 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 49.85
Current children cumulated vsize (KiB) 456544

[startup+25.5008 s]
/proc/loadavg: 7.71 7.98 7.99 9/186 29980
/proc/meminfo: memFree=25537180/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=604616 CPUtime=100.98 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 92282 0 0 0 10027 71 0 0 25 0 5 0 498557203 619126784 90761 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 151154 90761 176 36 0 149059 0
[pid=29971/tid=29973] ppid=29969 vsize=604616 CPUtime=25.18 cores=1,3,5,7
/proc/29971/task/29973/stat : 29973 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 15407 0 0 0 2502 16 0 0 25 0 5 0 498557231 619126784 90761 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=29971/tid=29974] ppid=29969 vsize=604616 CPUtime=25.2 cores=1,3,5,7
/proc/29971/task/29974/stat : 29974 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 20489 0 0 0 2502 18 0 0 25 0 5 0 498557231 619126784 90761 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29971/tid=29975] ppid=29969 vsize=604616 CPUtime=25.21 cores=1,3,5,7
/proc/29971/task/29975/stat : 29975 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 18446 0 0 0 2504 17 0 0 25 0 5 0 498557231 619126784 90761 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29971/tid=29976] ppid=29969 vsize=604616 CPUtime=25.09 cores=1,3,5,7
/proc/29971/task/29976/stat : 29976 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 29712 0 0 0 2491 18 0 0 25 0 5 0 498557231 619126784 90761 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 100.98
Current children cumulated vsize (KiB) 604616

[startup+51.1011 s]
/proc/loadavg: 7.72 7.95 7.98 9/186 29981
/proc/meminfo: memFree=24999976/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=711760 CPUtime=203.23 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 120215 0 0 0 20160 163 0 0 25 0 5 0 498557203 728842240 118694 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 177940 118694 176 36 0 175845 0
[pid=29971/tid=29973] ppid=29969 vsize=711760 CPUtime=50.78 cores=1,3,5,7
/proc/29971/task/29973/stat : 29973 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 22191 0 0 0 5038 40 0 0 25 0 5 0 498557231 728842240 118694 33554432000 4194304 4338100 140734570303184 18446744073709551615 4287761 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=29971/tid=29974] ppid=29969 vsize=711760 CPUtime=50.79 cores=1,3,5,7
/proc/29971/task/29974/stat : 29974 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 27517 0 0 0 5036 43 0 0 25 0 5 0 498557231 728842240 118694 33554432000 4194304 4338100 140734570303184 18446744073709551615 4240527 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29971/tid=29975] ppid=29969 vsize=711760 CPUtime=50.79 cores=1,3,5,7
/proc/29971/task/29975/stat : 29975 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 25359 0 0 0 5041 38 0 0 25 0 5 0 498557231 728842240 118694 33554432000 4194304 4338100 140734570303184 18446744073709551615 4287681 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29971/tid=29976] ppid=29969 vsize=711760 CPUtime=50.54 cores=1,3,5,7
/proc/29971/task/29976/stat : 29976 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 36920 0 0 0 5015 39 0 0 25 0 5 0 498557231 728842240 118694 33554432000 4194304 4338100 140734570303184 18446744073709551615 4256139 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.23
Current children cumulated vsize (KiB) 711760

[startup+102.301 s]
/proc/loadavg: 7.88 7.96 7.98 9/186 29982
/proc/meminfo: memFree=24380460/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=835324 CPUtime=407.65 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 150515 0 0 0 40415 350 0 0 25 0 5 0 498557203 855371776 148994 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 208831 148994 176 36 0 206736 0
[pid=29971/tid=29973] ppid=29969 vsize=835324 CPUtime=101.95 cores=1,3,5,7
/proc/29971/task/29973/stat : 29973 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 27544 0 0 0 10106 89 0 0 25 0 5 0 498557231 855371776 148994 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=29971/tid=29974] ppid=29969 vsize=835324 CPUtime=101.73 cores=1,3,5,7
/proc/29971/task/29974/stat : 29974 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 35683 0 0 0 10085 88 0 0 25 0 5 0 498557231 855371776 148994 33554432000 4194304 4338100 140734570303184 18446744073709551615 228094846389 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29971/tid=29975] ppid=29969 vsize=835324 CPUtime=101.97 cores=1,3,5,7
/proc/29971/task/29975/stat : 29975 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 33295 0 0 0 10113 84 0 0 25 0 5 0 498557231 855371776 148994 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29971/tid=29976] ppid=29969 vsize=835324 CPUtime=101.7 cores=1,3,5,7
/proc/29971/task/29976/stat : 29976 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 45765 0 0 0 10083 87 0 0 25 0 5 0 498557231 855371776 148994 33554432000 4194304 4338100 140734570303184 18446744073709551615 228094846702 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 407.65
Current children cumulated vsize (KiB) 835324

[startup+162.301 s]
/proc/loadavg: 7.95 7.96 7.98 9/186 29984
/proc/meminfo: memFree=23794232/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=972876 CPUtime=647.24 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 185278 0 0 0 64144 580 0 0 25 0 5 0 498557203 996225024 183757 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 243219 183757 176 36 0 241124 0
[pid=29971/tid=29973] ppid=29969 vsize=972876 CPUtime=161.93 cores=1,3,5,7
/proc/29971/task/29973/stat : 29973 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 37249 0 0 0 16045 148 0 0 25 0 5 0 498557231 996225024 183757 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=29971/tid=29974] ppid=29969 vsize=972876 CPUtime=161.4 cores=1,3,5,7
/proc/29971/task/29974/stat : 29974 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 45228 0 0 0 15994 146 0 0 25 0 5 0 498557231 996225024 183757 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29971/tid=29975] ppid=29969 vsize=972876 CPUtime=161.95 cores=1,3,5,7
/proc/29971/task/29975/stat : 29975 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 41862 0 0 0 16052 143 0 0 25 0 5 0 498557231 996225024 183757 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29971/tid=29976] ppid=29969 vsize=972876 CPUtime=161.67 cores=1,3,5,7
/proc/29971/task/29976/stat : 29976 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 52711 0 0 0 16025 142 0 0 25 0 5 0 498557231 996225024 183757 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 647.24
Current children cumulated vsize (KiB) 972876

[startup+222.3 s]
/proc/loadavg: 8.16 8.02 7.99 9/186 29986
/proc/meminfo: memFree=23342456/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=1063852 CPUtime=886.86 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 209034 0 0 0 87929 757 0 0 25 0 5 0 498557203 1089384448 207513 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 265963 207513 176 36 0 263868 0
[pid=29971/tid=29973] ppid=29969 vsize=1063852 CPUtime=221.91 cores=1,3,5,7
/proc/29971/task/29973/stat : 29973 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 42988 0 0 0 21998 193 0 0 25 0 5 0 498557231 1089384448 207513 33554432000 4194304 4338100 140734570303184 18446744073709551615 228095147752 0 0 4096 0 0 0 0 -1 1 0 0 0

################
# More data... #
################

/proc/29971/task/29976/stat : 29976 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 361152 0 0 0 69509 570 0 0 25 0 5 0 498557231 4354179072 1009734 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2801.63
Current children cumulated vsize (KiB) 4252128

[startup+762.301 s]
/proc/loadavg: 8.15 8.12 8.04 9/186 30003
/proc/meminfo: memFree=18611300/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=4390552 CPUtime=3041.19 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 1046102 0 0 0 302024 2095 0 0 25 0 5 0 498557203 4495925248 1044581 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 1097638 1044581 176 36 0 1095543 0
[pid=29971/tid=29973] ppid=29969 vsize=4390552 CPUtime=760.71 cores=1,3,5,7
/proc/29971/task/29973/stat : 29973 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 228886 0 0 0 75616 455 0 0 25 0 5 0 498557231 4495925248 1044581 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29971/tid=29974] ppid=29969 vsize=4390552 CPUtime=759 cores=1,3,5,7
/proc/29971/task/29974/stat : 29974 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 105710 0 0 0 75488 412 0 0 25 0 5 0 498557231 4495925248 1044581 33554432000 4194304 4338100 140734570303184 18446744073709551615 228094846420 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29971/tid=29975] ppid=29969 vsize=4390552 CPUtime=760.39 cores=1,3,5,7
/proc/29971/task/29975/stat : 29975 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 325659 0 0 0 75433 606 0 0 25 0 5 0 498557231 4495925248 1044581 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29971/tid=29976] ppid=29969 vsize=4390552 CPUtime=760.79 cores=1,3,5,7
/proc/29971/task/29976/stat : 29976 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 377619 0 0 0 75459 620 0 0 25 0 5 0 498557231 4495925248 1044581 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254439 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3041.19
Current children cumulated vsize (KiB) 4390552

[startup+822.301 s]
/proc/loadavg: 8.14 8.13 8.05 9/186 30005
/proc/meminfo: memFree=18261084/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=4546460 CPUtime=3280.64 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 1085405 0 0 0 325820 2244 0 0 25 0 5 0 498557203 4655575040 1083884 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 1136615 1083884 176 36 0 1134520 0
[pid=29971/tid=29973] ppid=29969 vsize=4546460 CPUtime=820.65 cores=1,3,5,7
/proc/29971/task/29973/stat : 29973 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 234299 0 0 0 81582 483 0 0 25 0 5 0 498557231 4655575040 1083884 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29971/tid=29974] ppid=29969 vsize=4546460 CPUtime=818.93 cores=1,3,5,7
/proc/29971/task/29974/stat : 29974 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 109584 0 0 0 81455 438 0 0 25 0 5 0 498557231 4655575040 1083884 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29971/tid=29975] ppid=29969 vsize=4546460 CPUtime=820.02 cores=1,3,5,7
/proc/29971/task/29975/stat : 29975 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 334478 0 0 0 81352 650 0 0 25 0 5 0 498557231 4655575040 1083884 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29971/tid=29976] ppid=29969 vsize=4546460 CPUtime=820.72 cores=1,3,5,7
/proc/29971/task/29976/stat : 29976 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 398816 0 0 0 81402 670 0 0 25 0 5 0 498557231 4655575040 1083884 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3280.64
Current children cumulated vsize (KiB) 4546460

[startup+882.3 s]
/proc/loadavg: 8.16 8.14 8.06 9/186 30006
/proc/meminfo: memFree=17771248/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=4728544 CPUtime=3520.07 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 1132254 0 0 0 349606 2401 0 0 25 0 5 0 498557203 4842029056 1130733 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 1182136 1130733 176 36 0 1180041 0
[pid=29971/tid=29973] ppid=29969 vsize=4728544 CPUtime=880.6 cores=1,3,5,7
/proc/29971/task/29973/stat : 29973 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 237204 0 0 0 87543 517 0 0 25 0 5 0 498557231 4842029056 1130733 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29971/tid=29974] ppid=29969 vsize=4728544 CPUtime=878.84 cores=1,3,5,7
/proc/29971/task/29974/stat : 29974 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 109737 0 0 0 87416 468 0 0 25 0 5 0 498557231 4842029056 1130733 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254326 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29971/tid=29975] ppid=29969 vsize=4728544 CPUtime=879.65 cores=1,3,5,7
/proc/29971/task/29975/stat : 29975 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 359681 0 0 0 87267 698 0 0 25 0 5 0 498557231 4842029056 1130733 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29971/tid=29976] ppid=29969 vsize=4728544 CPUtime=880.68 cores=1,3,5,7
/proc/29971/task/29976/stat : 29976 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 417404 0 0 0 87352 716 0 0 25 0 5 0 498557231 4842029056 1130733 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3520.07
Current children cumulated vsize (KiB) 4728544

[startup+942.3 s]
/proc/loadavg: 8.17 8.15 8.06 9/186 30008
/proc/meminfo: memFree=17392144/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=4939968 CPUtime=3759.68 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 1185602 0 0 0 373406 2562 0 0 25 0 5 0 498557203 5058527232 1184081 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 1234992 1184081 176 36 0 1232897 0
[pid=29971/tid=29973] ppid=29969 vsize=4939968 CPUtime=940.56 cores=1,3,5,7
/proc/29971/task/29973/stat : 29973 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 245742 0 0 0 93504 552 0 0 25 0 5 0 498557231 5058527232 1184081 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29971/tid=29974] ppid=29969 vsize=4939968 CPUtime=938.8 cores=1,3,5,7
/proc/29971/task/29974/stat : 29974 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 110145 0 0 0 93377 503 0 0 25 0 5 0 498557231 5058527232 1184081 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29971/tid=29975] ppid=29969 vsize=4939968 CPUtime=939.35 cores=1,3,5,7
/proc/29971/task/29975/stat : 29975 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 379343 0 0 0 93191 744 0 0 25 0 5 0 498557231 5058527232 1184081 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29971/tid=29976] ppid=29969 vsize=4939968 CPUtime=940.65 cores=1,3,5,7
/proc/29971/task/29976/stat : 29976 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 442144 0 0 0 93304 761 0 0 25 0 5 0 498557231 5058527232 1184081 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3759.68
Current children cumulated vsize (KiB) 4939968

[startup+1002.3 s]
/proc/loadavg: 8.18 8.15 8.07 9/186 30009
/proc/meminfo: memFree=16951148/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=5318756 CPUtime=3998.93 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 1245649 0 0 0 397169 2724 0 0 25 0 5 0 498557203 5446406144 1244128 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 1329689 1244128 176 36 0 1327594 0
[pid=29971/tid=29973] ppid=29969 vsize=5318756 CPUtime=1000.38 cores=1,3,5,7
/proc/29971/task/29973/stat : 29973 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 263396 0 0 0 99450 588 0 0 25 0 5 0 498557231 5446406144 1244128 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29971/tid=29974] ppid=29969 vsize=5318756 CPUtime=998.72 cores=1,3,5,7
/proc/29971/task/29974/stat : 29974 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 114416 0 0 0 99335 537 0 0 25 0 5 0 498557231 5446406144 1244128 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29971/tid=29975] ppid=29969 vsize=5318756 CPUtime=998.96 cores=1,3,5,7
/proc/29971/task/29975/stat : 29975 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 400607 0 0 0 99107 789 0 0 25 0 5 0 498557231 5446406144 1244128 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29971/tid=29976] ppid=29969 vsize=5318756 CPUtime=1000.55 cores=1,3,5,7
/proc/29971/task/29976/stat : 29976 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 459002 0 0 0 99248 807 0 0 25 0 5 0 498557231 5446406144 1244128 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3998.93
Current children cumulated vsize (KiB) 5318756

[startup+1062.3 s]
/proc/loadavg: 8.06 8.12 8.06 9/186 30011
/proc/meminfo: memFree=16915784/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=5328404 CPUtime=4238.62 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 1248349 0 0 0 421060 2802 0 0 25 0 5 0 498557203 5456285696 1246828 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 1332101 1246828 176 36 0 1330006 0
[pid=29971/tid=29973] ppid=29969 vsize=5328404 CPUtime=1060.38 cores=1,3,5,7
/proc/29971/task/29973/stat : 29973 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 263828 0 0 0 105430 608 0 0 25 0 5 0 498557231 5456285696 1246828 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29971/tid=29974] ppid=29969 vsize=5328404 CPUtime=1058.72 cores=1,3,5,7
/proc/29971/task/29974/stat : 29974 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 115095 0 0 0 105313 559 0 0 25 0 5 0 498557231 5456285696 1246828 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107276660 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29971/tid=29975] ppid=29969 vsize=5328404 CPUtime=1058.66 cores=1,3,5,7
/proc/29971/task/29975/stat : 29975 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 401112 0 0 0 105057 809 0 0 25 0 5 0 498557231 5456285696 1246828 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29971/tid=29976] ppid=29969 vsize=5328404 CPUtime=1060.54 cores=1,3,5,7
/proc/29971/task/29976/stat : 29976 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 460086 0 0 0 105230 824 0 0 25 0 5 0 498557231 5456285696 1246828 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4238.62
Current children cumulated vsize (KiB) 5328404

[startup+1122.3 s]
/proc/loadavg: 7.67 8.00 8.02 9/186 30013
/proc/meminfo: memFree=16730744/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=5328668 CPUtime=4478.34 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 1249269 0 0 0 444956 2878 0 0 25 0 5 0 498557203 5456556032 1247748 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 1332167 1247748 176 36 0 1330072 0
[pid=29971/tid=29973] ppid=29969 vsize=5328668 CPUtime=1120.39 cores=1,3,5,7
/proc/29971/task/29973/stat : 29973 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 263886 0 0 0 111410 629 0 0 25 0 5 0 498557231 5456556032 1247748 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29971/tid=29974] ppid=29969 vsize=5328668 CPUtime=1118.72 cores=1,3,5,7
/proc/29971/task/29974/stat : 29974 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 115840 0 0 0 111292 580 0 0 25 0 5 0 498557231 5456556032 1247748 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29971/tid=29975] ppid=29969 vsize=5328668 CPUtime=1118.36 cores=1,3,5,7
/proc/29971/task/29975/stat : 29975 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 401170 0 0 0 111012 824 0 0 25 0 5 0 498557231 5456556032 1247748 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29971/tid=29976] ppid=29969 vsize=5328668 CPUtime=1120.55 cores=1,3,5,7
/proc/29971/task/29976/stat : 29976 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 460145 0 0 0 111213 842 0 0 25 0 5 0 498557231 5456556032 1247748 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4478.34
Current children cumulated vsize (KiB) 5328668

[startup+1182.3 s]
/proc/loadavg: 7.91 8.01 8.02 9/186 30014
/proc/meminfo: memFree=16604980/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=5347700 CPUtime=4717.95 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) S 29969 29971 29500 0 -1 4202496 1255113 0 0 0 468828 2967 0 0 25 0 5 0 498557203 5476044800 1253592 33554432000 4194304 4338100 140734570303184 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 1336925 1253592 176 36 0 1334830 0
[pid=29971/tid=29973] ppid=29969 vsize=5347700 CPUtime=1180.35 cores=1,3,5,7
/proc/29971/task/29973/stat : 29973 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 263955 0 0 0 117382 653 0 0 25 0 5 0 498557231 5476044800 1253592 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29971/tid=29974] ppid=29969 vsize=5347700 CPUtime=1178.69 cores=1,3,5,7
/proc/29971/task/29974/stat : 29974 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 115909 0 0 0 117265 604 0 0 25 0 5 0 498557231 5476044800 1253592 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29971/tid=29975] ppid=29969 vsize=5347700 CPUtime=1178.07 cores=1,3,5,7
/proc/29971/task/29975/stat : 29975 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 405662 0 0 0 116960 847 0 0 25 0 5 0 498557231 5476044800 1253592 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29971/tid=29976] ppid=29969 vsize=5347700 CPUtime=1180.53 cores=1,3,5,7
/proc/29971/task/29976/stat : 29976 (ttsth-5-4) R 29969 29971 29500 0 -1 4202560 461359 0 0 0 117192 861 0 0 25 0 5 0 498557231 5476044800 1253592 33554432000 4194304 4338100 140734570303184 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4717.95
Current children cumulated vsize (KiB) 5347700



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.9 s]
/proc/loadavg: 8.00 8.03 8.03 6/183 30015
/proc/meminfo: memFree=16944472/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=0 CPUtime=4800.18 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) Z 29969 29971 29500 0 -1 4203548 1256616 0 0 0 477023 2995 0 0 18 0 2 0 498557203 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 0 0 0 0 0 0 0
[pid=29971/tid=29974] ppid=29969 vsize=0 CPUtime=1199.3 cores=1,3,5,7
/proc/29971/task/29974/stat : 29974 (ttsth-5-4) R 29969 29971 29500 0 -1 4203588 115928 0 0 0 119317 613 0 0 25 0 2 0 498557231 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.18
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.01 s]
/proc/loadavg: 8.00 8.03 8.03 6/183 30015
/proc/meminfo: memFree=16944472/32951124 swapFree=67111524/67111528
[pid=29971] ppid=29969 vsize=0 CPUtime=4800.29 cores=1,3,5,7
/proc/29971/stat : 29971 (ttsth-5-4) Z 29969 29971 29500 0 -1 4203548 1256616 0 0 0 477023 3006 0 0 18 0 2 0 498557203 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29971/statm: 0 0 0 0 0 0 0
[pid=29971/tid=29974] ppid=29969 vsize=0 CPUtime=1199.41 cores=1,3,5,7
/proc/29971/task/29974/stat : 29974 (ttsth-5-4) R 29969 29971 29500 0 -1 4203588 115928 0 0 0 119317 624 0 0 25 0 2 0 498557231 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.29
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1203.09
CPU time (s): 4800.37
CPU user time (s): 4770.24
CPU system time (s): 30.1374
CPU usage (%): 399.004
Max. virtual memory (cumulated for all children) (KiB): 5354088

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4770.24
system time used= 30.1374
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1256616
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 101711
involuntary context switches= 28352

runsolver used 1.76473 second user time and 4.19036 second system time

The end

Launcher Data 

Begin job on node142 at 2011-04-29 09:16:57
IDJOB=3249284
IDBENCH=83163
IDSOLVER=1654
FILE ID=node142/3249284-1304061417
RUNJOBID= node142-1304060210-29816
PBS_JOBID= 13170434
Free space on /tmp= 71312 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/mosoi/289/289-unsat-5x41.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3249284-1304061417/watcher-3249284-1304061417 -o /tmp/evaluation-result-3249284-1304061417/solver-3249284-1304061417 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3249284-1304061417.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 420773b7b438dd18c8a88a15fb5bcae1
RANDOM SEED=1309523383

node142.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5131.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      26584200 kB
Buffers:       1653628 kB
Cached:        4102796 kB
SwapCached:          4 kB
Active:         692368 kB
Inactive:      5135668 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      26584200 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:            1020 kB
Writeback:          80 kB
AnonPages:       71684 kB
Mapped:          15332 kB
Slab:           475692 kB
PageTables:       4828 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   211304 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71312 MiB
End job on node142 at 2011-04-29 09:37:02