Trace number 3274250

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.15 1215.01

General information on the benchmark

Namecrafted/spence/sgen/
sgen3-n190-s29985700-unsat.cnf
MD5SUMf2cf61fc47972fb7e89be65d1a83332e
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables193
Number of clauses396
Sum of the clauses size1188
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3396
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3274250-1304060238.cnf 
0.00/0.00	c Using SATTIMEOUT of 4800
0.00/0.00	c Using SATRAM of 15500
0.00/0.00	c Size of subset node = 32
0.00/0.00	c Variables = 193
0.00/0.00	c Clauses = 396
0.00/0.00	c Literals = 1188
0.00/0.00	c Reduced size:
0.00/0.00	c Variables = 193
0.00/0.00	c Clauses = 396
0.00/0.00	c Literals = 1188
0.00/0.00	c a2
0.00/0.00	c a1
2.30/2.39	c Independent sub-problems = 1
2.30/2.39	c Sub-problem with 193 variables and 396 clauses
2.30/2.39	c using permutation method 3
2.30/2.39	c Ordering finished 2.390 seconds
2.30/2.39	c mapped 2.390 seconds
2.30/2.39	c permuted 2.390 seconds
2.30/2.39	c about to sort
2.30/2.39	c sorted
2.30/2.39	c clauses now 396
2.30/2.39	c init finished 2.390 seconds
2.30/2.39	c tree built 2.390 seconds
2.30/2.39	c 
2.30/2.39	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
2.30/2.39	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
2.30/2.39	c ................................ | ..........                    |                                  2.4         0.0             
2.30/2.39	c ................................ | ..........                  1 |                                  2.4         0.4             
2.30/2.39	c ................................ | ..........                  2 |                                  2.4         0.8             
2.30/2.39	c ................................ | ..........                  3 |                                  2.4         1.3             
2.30/2.39	c ................................ | ..........                  4 |                                  2.4         1.7             
2.30/2.39	c ................................ | ..........                  5 |                                  2.4         2.1             
2.30/2.40	c ................................ | ..........               1000 | 31           201                 2.4       416.7         4760
95.81/25.95	c ................................ | ..........            1904000 | 270594       338318             96.0     19831.3     10261386
191.75/50.16	c ................................ | ..........            3223000 | 476820       493970            192.0     16785.6     11847579
287.71/74.38	c ................................ | ..........            4229000 | 647723       587440            288.1     14681.0     13882796
383.83/98.55	c ................................ | ..........            5045000 | 783067       633862            384.0     13136.7     15505522
479.86/122.79	c ................................ | *.........            5761000 | 908805       774917            480.2     11998.1     17561744
576.11/147.01	c ................................ | *.........            6535000 | 1014393      856668            576.2     11341.7     19496206
671.67/171.28	c ................................ | *.........            7523000 | 1150624      956338            672.0     11194.9     20300961
767.69/195.48	c ................................ | *.........            8521000 | 1313047      1023595           768.0     11094.6     20300961
863.94/219.63	c ................................ | *.........            9261000 | 1447696      1073599           864.1     10717.9     20769630
960.06/243.90	c ................................ | **........           10096000 | 1608959      1131215           960.1     10515.9     21838836
1055.86/268.14	c ................................ | **........           10948000 | 1772840      1188372          1056.0     10366.9     22390017
1151.71/292.48	c ................................ | **........           12004000 | 1956013      1290493          1152.1     10419.6     22755160
1247.69/316.68	c ................................ | **........           12870000 | 2100683      1407137          1248.0     10312.2     24125464
1343.94/340.84	c ................................ | **........           13875000 | 2226352      1557543          1344.1     10322.9     25409692
1440.05/365.14	c ................................ | ***.......           14641000 | 2361187      1635684          1440.2     10165.9     26401715
1535.69/389.38	c ................................ | ***.......           15477000 | 2513672      1722004          1536.0     10076.2     27250337
1632.01/413.71	c ................................ | ***.......           16295000 | 2662570      1798813          1632.0      9984.4     27650147
1727.77/437.96	c ................................ | ***.......           17226000 | 2831668      1867232          1728.0      9968.6     28465226
1823.84/462.25	c ................................ | ***.......           18049000 | 2966288      1962843          1824.0      9895.0     29339464
1919.96/486.51	c ................................ | ****......           18880000 | 3086315      2043330          1920.0      9833.2     30197612
2015.63/510.79	c ................................ | ****......           19444000 | 3200975      2082119          2016.0      9644.7     30955336
2111.95/535.03	c ................................ | ****......           20192000 | 3358796      2105930          2112.1      9560.2     31161827
2207.81/559.35	c ................................ | ****......           21288000 | 3556059      2170712          2208.0      9641.3     31271792
2303.80/583.55	c ................................ | ****......           22039000 | 3690361      2225559          2304.0      9565.5     31399720
2399.98/607.91	c ................................ | *****.....           23163000 | 3910679      2268814          2400.0      9651.2     32172382
2496.01/632.30	c ................................ | *****.....           24059000 | 4081713      2306522          2496.0      9638.9     32528454
2591.68/656.58	c ................................ | *****.....           24789000 | 4224866      2342980          2592.0      9563.6     33126775
2687.97/681.02	c ................................ | *****.....           25528000 | 4370300      2394668          2688.1      9496.8     33686411
2783.73/705.27	c ................................ | *****.....           26364000 | 4525148      2425824          2784.0      9469.8     34664930
2879.79/729.57	c ................................ | ******....           27194000 | 4680437      2451529          2880.1      9442.1     34664930
2975.85/753.73	c ................................ | ******....           28076000 | 4852740      2488431          2976.0      9434.1     35380332
3071.79/777.96	c ................................ | ******....           28953000 | 5015291      2532614          3072.0      9424.7     35805868
3168.04/802.11	c ................................ | ******....           29586000 | 5146911      2561037          3168.1      9338.8     36559859
3263.73/826.37	c ................................ | ******....           30176000 | 5268080      2586918          3264.0      9245.0     36619502
3360.05/850.63	c ................................ | *******...           30755000 | 5384486      2628222          3360.2      9152.8     36826212
3455.85/874.85	c ................................ | *******...           31229000 | 5483557      2653783          3456.0      9036.1     36826212
3551.83/899.14	c ................................ | *******...           32079000 | 5638479      2718978          3552.0      9031.2     36972618
3647.67/923.39	c ................................ | *******...           32727000 | 5778183      2759282          3648.0      8971.1     37205058
3744.02/947.61	c ................................ | *******...           33621000 | 5944275      2800688          3744.1      8979.8     38338827
3839.87/971.85	c ................................ | ********..           34312000 | 6073009      2835393          3840.1      8935.2     39515156
3935.94/996.24	c ................................ | ********..           35011000 | 6223253      2885229          3936.1      8894.8     39515156
4031.97/1020.41	c ................................ | ********..           35761000 | 6370864      2939314          4032.0      8869.3     40296662
4127.84/1044.65	c ................................ | ********..           36455000 | 6487565      3006996          4128.1      8831.0     40567085
4223.69/1068.88	c ................................ | ********..           37177000 | 6642415      3033693          4224.0      8801.3     40701857
4320.20/1093.33	c ................................ | *********.           38040000 | 6819846      3067503          4320.3      8804.9     41454914
4416.00/1117.72	c ................................ | *********.           38964000 | 7002724      3129109          4416.1      8823.2     44640426
4511.84/1142.06	c ................................ | *********.           39527000 | 7115738      3169313          4512.1      8760.3     46766037
4607.68/1166.28	c ................................ | *********.           39948000 | 7200858      3208602          4608.0      8669.2     46954519
4703.76/1190.66	c ................................ | *********.           40641000 | 7315155      3261511          4704.0      8639.6     46954519
4799.94/1214.93	c CPU limit (4800) reached
4799.94/1214.93	c Time = 4800.08 seconds
4799.94/1214.93	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3274250-1304060238/watcher-3274250-1304060238 -o /tmp/evaluation-result-3274250-1304060238/solver-3274250-1304060238 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3274250-1304060238.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.05 8.05 8.01 6/181 9640
/proc/meminfo: memFree=17961748/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=8840 CPUtime=0 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) R 9638 9640 9362 0 -1 4202496 285 0 0 0 0 0 0 0 25 0 1 0 498439942 9052160 227 33554432000 4194304 4338100 140735553752512 18446744073709551615 4290761 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/9640/statm: 2210 227 163 36 0 115 0

[startup+0.0873681 s]
/proc/loadavg: 8.05 8.05 8.01 6/181 9640
/proc/meminfo: memFree=17961748/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=8840 CPUtime=0.08 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) R 9638 9640 9362 0 -1 4202496 286 0 0 0 8 0 0 0 25 0 1 0 498439942 9052160 228 33554432000 4194304 4338100 140735553752512 18446744073709551615 4290917 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/9640/statm: 2210 228 163 36 0 115 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 8840

[startup+0.100374 s]
/proc/loadavg: 8.05 8.05 8.01 6/181 9640
/proc/meminfo: memFree=17961748/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=8840 CPUtime=0.09 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) R 9638 9640 9362 0 -1 4202496 286 0 0 0 9 0 0 0 25 0 1 0 498439942 9052160 228 33554432000 4194304 4338100 140735553752512 18446744073709551615 255106433947 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/9640/statm: 2210 228 163 36 0 115 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 8840

[startup+0.30034 s]
/proc/loadavg: 8.05 8.05 8.01 6/181 9640
/proc/meminfo: memFree=17961748/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=8840 CPUtime=0.29 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) R 9638 9640 9362 0 -1 4202496 286 0 0 0 29 0 0 0 25 0 1 0 498439942 9052160 228 33554432000 4194304 4338100 140735553752512 18446744073709551615 4290842 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/9640/statm: 2210 228 163 36 0 115 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 8840

[startup+0.700256 s]
/proc/loadavg: 8.05 8.05 8.01 6/181 9640
/proc/meminfo: memFree=17961748/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=8840 CPUtime=0.69 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) R 9638 9640 9362 0 -1 4202496 296 0 0 0 69 0 0 0 25 0 1 0 498439942 9052160 238 33554432000 4194304 4338100 140735553752512 18446744073709551615 4291035 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/9640/statm: 2210 238 173 36 0 115 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 8840

[startup+1.50012 s]
/proc/loadavg: 7.81 8.00 7.99 7/182 9641
/proc/meminfo: memFree=17957780/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=8840 CPUtime=1.49 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) R 9638 9640 9362 0 -1 4202496 296 0 0 0 149 0 0 0 25 0 1 0 498439942 9052160 238 33554432000 4194304 4338100 140735553752512 18446744073709551615 4290979 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/9640/statm: 2210 238 173 36 0 115 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 8840

[startup+3.1008 s]
/proc/loadavg: 7.81 8.00 7.99 7/182 9641
/proc/meminfo: memFree=17952200/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=217668 CPUtime=5.19 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) S 9638 9640 9362 0 -1 4202496 8239 0 0 0 517 2 0 0 25 0 5 0 498439942 222892032 8181 33554432000 4194304 4338100 140735553752512 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 54417 8181 196 36 0 52322 0
Current children cumulated CPU time (s) 5.19
Current children cumulated vsize (KiB) 217668

[startup+6.30024 s]
/proc/loadavg: 7.81 8.00 7.99 10/186 9645
/proc/meminfo: memFree=17855204/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=291632 CPUtime=17.94 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) S 9638 9640 9362 0 -1 4202496 26323 0 0 0 1789 5 0 0 25 0 5 0 498439942 298631168 26257 33554432000 4194304 4338100 140735553752512 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 72908 26257 198 36 0 70813 0
[pid=9640/tid=9642] ppid=9638 vsize=291632 CPUtime=3.89 cores=1,3,5,7
/proc/9640/task/9642/stat : 9642 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 4414 0 0 0 387 2 0 0 25 0 5 0 498440182 298631168 26257 33554432000 4194304 4338100 140735553752512 18446744073709551615 255106164462 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9640/tid=9643] ppid=9638 vsize=291632 CPUtime=3.87 cores=1,3,5,7
/proc/9640/task/9643/stat : 9643 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 6578 0 0 0 386 1 0 0 25 0 5 0 498440182 298631168 26257 33554432000 4194304 4338100 140735553752512 18446744073709551615 4256127 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9640/tid=9644] ppid=9638 vsize=291632 CPUtime=3.89 cores=1,3,5,7
/proc/9640/task/9644/stat : 9644 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 7107 0 0 0 388 1 0 0 25 0 5 0 498440182 298631168 26257 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254285 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9640/tid=9645] ppid=9638 vsize=291632 CPUtime=3.88 cores=1,3,5,7
/proc/9640/task/9645/stat : 9645 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 7366 0 0 0 387 1 0 0 25 0 5 0 498440182 298631168 26257 33554432000 4194304 4338100 140735553752512 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 17.94
Current children cumulated vsize (KiB) 291632

[startup+12.7011 s]
/proc/loadavg: 7.91 8.01 8.00 10/186 9645
/proc/meminfo: memFree=17745336/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=370480 CPUtime=43.37 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) S 9638 9640 9362 0 -1 4202496 46953 0 0 0 4326 11 0 0 25 0 5 0 498439942 379371520 46887 33554432000 4194304 4338100 140735553752512 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 92620 46887 198 36 0 90525 0
[pid=9640/tid=9642] ppid=9638 vsize=370480 CPUtime=10.22 cores=1,3,5,7
/proc/9640/task/9642/stat : 9642 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 8047 0 0 0 1019 3 0 0 25 0 5 0 498440182 379371520 46887 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9640/tid=9643] ppid=9638 vsize=370480 CPUtime=10.2 cores=1,3,5,7
/proc/9640/task/9643/stat : 9643 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 9973 0 0 0 1019 1 0 0 25 0 5 0 498440182 379371520 46887 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9640/tid=9644] ppid=9638 vsize=370480 CPUtime=10.26 cores=1,3,5,7
/proc/9640/task/9644/stat : 9644 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 12044 0 0 0 1024 2 0 0 25 0 5 0 498440182 379371520 46887 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9640/tid=9645] ppid=9638 vsize=370480 CPUtime=10.26 cores=1,3,5,7
/proc/9640/task/9645/stat : 9645 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 16031 0 0 0 1023 3 0 0 25 0 5 0 498440182 379371520 46887 33554432000 4194304 4338100 140735553752512 18446744073709551615 255106163821 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 43.37
Current children cumulated vsize (KiB) 370480

[startup+25.5009 s]
/proc/loadavg: 7.85 8.00 7.99 10/186 9646
/proc/meminfo: memFree=17579328/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=515008 CPUtime=94.21 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) S 9638 9640 9362 0 -1 4202496 82138 0 0 0 9398 23 0 0 25 0 5 0 498439942 527368192 82072 33554432000 4194304 4338100 140735553752512 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 128752 82072 198 36 0 126657 0
[pid=9640/tid=9642] ppid=9638 vsize=515008 CPUtime=22.92 cores=1,3,5,7
/proc/9640/task/9642/stat : 9642 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 14754 0 0 0 2285 7 0 0 25 0 5 0 498440182 527368192 82072 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254337 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9640/tid=9643] ppid=9638 vsize=515008 CPUtime=22.93 cores=1,3,5,7
/proc/9640/task/9643/stat : 9643 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 16430 0 0 0 2290 3 0 0 25 0 5 0 498440182 527368192 82072 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9640/tid=9644] ppid=9638 vsize=515008 CPUtime=23 cores=1,3,5,7
/proc/9640/task/9644/stat : 9644 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 21254 0 0 0 2295 5 0 0 25 0 5 0 498440182 527368192 82072 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9640/tid=9645] ppid=9638 vsize=515008 CPUtime=22.94 cores=1,3,5,7
/proc/9640/task/9645/stat : 9645 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 28842 0 0 0 2288 6 0 0 25 0 5 0 498440182 527368192 82072 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 94.21
Current children cumulated vsize (KiB) 515008

[startup+51.1004 s]
/proc/loadavg: 7.90 8.00 7.99 10/186 9646
/proc/meminfo: memFree=17459160/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=592560 CPUtime=195.72 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) S 9638 9640 9362 0 -1 4202496 100834 0 0 0 19537 35 0 0 25 0 5 0 498439942 606781440 100768 33554432000 4194304 4338100 140735553752512 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 148140 100768 198 36 0 146045 0
[pid=9640/tid=9642] ppid=9638 vsize=592560 CPUtime=48.24 cores=1,3,5,7
/proc/9640/task/9642/stat : 9642 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 19496 0 0 0 4813 11 0 0 25 0 5 0 498440182 606781440 100768 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9640/tid=9643] ppid=9638 vsize=592560 CPUtime=48.38 cores=1,3,5,7
/proc/9640/task/9643/stat : 9643 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 19320 0 0 0 4832 6 0 0 25 0 5 0 498440182 606781440 100769 33554432000 4194304 4338100 140735553752512 18446744073709551615 4240537 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9640/tid=9644] ppid=9638 vsize=592560 CPUtime=48.39 cores=1,3,5,7
/proc/9640/task/9644/stat : 9644 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 23533 0 0 0 4833 6 0 0 25 0 5 0 498440182 606781440 100769 33554432000 4194304 4338100 140735553752512 18446744073709551615 255106163821 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9640/tid=9645] ppid=9638 vsize=592560 CPUtime=48.29 cores=1,3,5,7
/proc/9640/task/9645/stat : 9645 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 37627 0 0 0 4818 11 0 0 25 0 5 0 498440182 606781440 100769 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254254 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 195.72
Current children cumulated vsize (KiB) 592560

[startup+102.306 s]
/proc/loadavg: 7.96 8.00 7.99 10/186 9648
/proc/meminfo: memFree=17242896/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=740944 CPUtime=399 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) S 9638 9640 9362 0 -1 4202496 137800 0 0 0 39840 60 0 0 25 0 5 0 498439942 758726656 137734 33554432000 4194304 4338100 140735553752512 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 185236 137734 198 36 0 183141 0
[pid=9640/tid=9642] ppid=9638 vsize=740944 CPUtime=98.86 cores=1,3,5,7
/proc/9640/task/9642/stat : 9642 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 30358 0 0 0 9865 21 0 0 25 0 5 0 498440182 758726656 137735 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254326 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9640/tid=9643] ppid=9638 vsize=740944 CPUtime=99.3 cores=1,3,5,7
/proc/9640/task/9643/stat : 9643 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 23263 0 0 0 9922 8 0 0 25 0 5 0 498440182 758726656 137735 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9640/tid=9644] ppid=9638 vsize=740944 CPUtime=99.28 cores=1,3,5,7
/proc/9640/task/9644/stat : 9644 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 29679 0 0 0 9919 9 0 0 25 0 5 0 498440182 758726656 137735 33554432000 4194304 4338100 140735553752512 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9640/tid=9645] ppid=9638 vsize=740944 CPUtime=99.12 cores=1,3,5,7
/proc/9640/task/9645/stat : 9645 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 53643 0 0 0 9893 19 0 0 25 0 5 0 498440182 758726656 137735 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 399
Current children cumulated vsize (KiB) 740944

[startup+162.301 s]
/proc/loadavg: 8.02 8.01 8.00 10/186 9649
/proc/meminfo: memFree=17004684/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=874584 CPUtime=636.34 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) S 9638 9640 9362 0 -1 4202496 171266 0 0 0 63551 83 0 0 25 0 5 0 498439942 895574016 171200 33554432000 4194304 4338100 140735553752512 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 218646 171200 198 36 0 216551 0
[pid=9640/tid=9642] ppid=9638 vsize=874584 CPUtime=158.28 cores=1,3,5,7
/proc/9640/task/9642/stat : 9642 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 36344 0 0 0 15800 28 0 0 25 0 5 0 498440182 895574016 171200 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9640/tid=9643] ppid=9638 vsize=874584 CPUtime=158.52 cores=1,3,5,7
/proc/9640/task/9643/stat : 9643 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 30145 0 0 0 15839 13 0 0 25 0 5 0 498440182 895574016 171200 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254407 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9640/tid=9644] ppid=9638 vsize=874584 CPUtime=158.52 cores=1,3,5,7
/proc/9640/task/9644/stat : 9644 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 38674 0 0 0 15839 13 0 0 25 0 5 0 498440182 895574016 171200 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9640/tid=9645] ppid=9638 vsize=874584 CPUtime=158.58 cores=1,3,5,7
/proc/9640/task/9645/stat : 9645 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 65245 0 0 0 15832 26 0 0 25 0 5 0 498440182 895574016 171200 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 636.34
Current children cumulated vsize (KiB) 874584

[startup+222.3 s]
/proc/loadavg: 8.04 8.02 8.00 10/186 9651
/proc/meminfo: memFree=16902124/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=919592 CPUtime=874.7 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) S 9638 9640 9362 0 -1 4202496 180227 0 0 0 87380 90 0 0 25 0 5 0 498439942 941662208 180161 33554432000 4194304 4338100 140735553752512 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 229898 180161 198 36 0 227803 0
[pid=9640/tid=9642] ppid=9638 vsize=919592 CPUtime=217.72 cores=1,3,5,7
/proc/9640/task/9642/stat : 9642 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 41432 0 0 0 21739 33 0 0 25 0 5 0 498440182 941662208 180161 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9640/tid=9643] ppid=9638 vsize=919592 CPUtime=218.13 cores=1,3,5,7
/proc/9640/task/9643/stat : 9643 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 31390 0 0 0 21799 14 0 0 25 0 5 0 498440182 941662208 180161 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9640/tid=9644] ppid=9638 vsize=919592 CPUtime=218.15 cores=1,3,5,7
/proc/9640/task/9644/stat : 9644 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 40049 0 0 0 21801 14 0 0 25 0 5 0 498440182 941662208 180161 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9640/tid=9645] ppid=9638 vsize=919592 CPUtime=218.26 cores=1,3,5,7
/proc/9640/task/9645/stat : 9645 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 66498 0 0 0 21799 27 0 0 25 0 5 0 498440182 941662208 180161 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 874.7
Current children cumulated vsize (KiB) 919592

################
# More data... #
################

[startup+702.301 s]
/proc/loadavg: 8.19 8.10 8.02 10/186 9673
/proc/meminfo: memFree=15861100/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=1439064 CPUtime=2772.27 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) S 9638 9640 9362 0 -1 4202496 307965 0 0 0 277000 227 0 0 25 0 5 0 498439942 1473601536 307899 33554432000 4194304 4338100 140735553752512 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 359766 307899 198 36 0 357671 0
[pid=9640/tid=9642] ppid=9638 vsize=1439064 CPUtime=689.66 cores=1,3,5,7
/proc/9640/task/9642/stat : 9642 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 116997 0 0 0 68857 109 0 0 25 0 5 0 498440182 1473601536 307899 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9640/tid=9643] ppid=9638 vsize=1439064 CPUtime=693.41 cores=1,3,5,7
/proc/9640/task/9643/stat : 9643 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 51091 0 0 0 69303 38 0 0 25 0 5 0 498440182 1473601536 307899 33554432000 4194304 4338100 140735553752512 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9640/tid=9644] ppid=9638 vsize=1439064 CPUtime=693.05 cores=1,3,5,7
/proc/9640/task/9644/stat : 9644 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 52992 0 0 0 69278 27 0 0 25 0 5 0 498440182 1473601536 307899 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9640/tid=9645] ppid=9638 vsize=1439064 CPUtime=693.71 cores=1,3,5,7
/proc/9640/task/9645/stat : 9645 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 86027 0 0 0 69320 51 0 0 25 0 5 0 498440182 1473601536 307899 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2772.27
Current children cumulated vsize (KiB) 1439064

[startup+762.306 s]
/proc/loadavg: 8.07 8.08 8.02 10/186 9674
/proc/meminfo: memFree=15768460/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=1464600 CPUtime=3009.94 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) S 9638 9640 9362 0 -1 4202496 318667 0 0 0 300757 237 0 0 25 0 5 0 498439942 1499750400 318601 33554432000 4194304 4338100 140735553752512 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 366150 318601 198 36 0 364055 0
[pid=9640/tid=9642] ppid=9638 vsize=1464600 CPUtime=748.68 cores=1,3,5,7
/proc/9640/task/9642/stat : 9642 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 123534 0 0 0 74751 117 0 0 25 0 5 0 498440182 1499750400 318601 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9640/tid=9643] ppid=9638 vsize=1464600 CPUtime=752.91 cores=1,3,5,7
/proc/9640/task/9643/stat : 9643 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 51149 0 0 0 75252 39 0 0 25 0 5 0 498440182 1499750400 318601 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9640/tid=9644] ppid=9638 vsize=1464600 CPUtime=752.59 cores=1,3,5,7
/proc/9640/task/9644/stat : 9644 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 55418 0 0 0 75231 28 0 0 25 0 5 0 498440182 1499750400 318601 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9640/tid=9645] ppid=9638 vsize=1464600 CPUtime=753.34 cores=1,3,5,7
/proc/9640/task/9645/stat : 9645 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 87708 0 0 0 75282 52 0 0 25 0 5 0 498440182 1499750400 318601 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3009.94
Current children cumulated vsize (KiB) 1464600

[startup+822.301 s]
/proc/loadavg: 8.07 8.08 8.02 10/186 9676
/proc/meminfo: memFree=15696144/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=1584312 CPUtime=3247.82 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) S 9638 9640 9362 0 -1 4202496 329843 0 0 0 324532 250 0 0 25 0 5 0 498439942 1622335488 329777 33554432000 4194304 4338100 140735553752512 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 396078 329777 198 36 0 393983 0
[pid=9640/tid=9642] ppid=9638 vsize=1584312 CPUtime=808.14 cores=1,3,5,7
/proc/9640/task/9642/stat : 9642 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 132681 0 0 0 80691 123 0 0 25 0 5 0 498440182 1622335488 329777 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9640/tid=9643] ppid=9638 vsize=1584312 CPUtime=812.3 cores=1,3,5,7
/proc/9640/task/9643/stat : 9643 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 51255 0 0 0 81190 40 0 0 25 0 5 0 498440182 1622335488 329777 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9640/tid=9644] ppid=9638 vsize=1584312 CPUtime=811.91 cores=1,3,5,7
/proc/9640/task/9644/stat : 9644 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 56633 0 0 0 81160 31 0 0 25 0 5 0 498440182 1622335488 329777 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9640/tid=9645] ppid=9638 vsize=1584312 CPUtime=813.04 cores=1,3,5,7
/proc/9640/task/9645/stat : 9645 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 88416 0 0 0 81250 54 0 0 25 0 5 0 498440182 1622335488 329777 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3247.82
Current children cumulated vsize (KiB) 1584312

[startup+882.301 s]
/proc/loadavg: 5.91 7.50 7.82 5/176 9696
/proc/meminfo: memFree=18227196/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=1591080 CPUtime=3485.54 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) S 9638 9640 9362 0 -1 4202496 333242 0 0 0 348294 260 0 0 25 0 5 0 498439942 1629265920 333176 33554432000 4194304 4338100 140735553752512 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 397770 333176 198 36 0 395675 0
[pid=9640/tid=9642] ppid=9638 vsize=1591080 CPUtime=867.48 cores=1,3,5,7
/proc/9640/task/9642/stat : 9642 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 134414 0 0 0 86620 128 0 0 25 0 5 0 498440182 1629265920 333176 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9640/tid=9643] ppid=9638 vsize=1591080 CPUtime=871.71 cores=1,3,5,7
/proc/9640/task/9643/stat : 9643 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 51425 0 0 0 87130 41 0 0 25 0 5 0 498440182 1629265920 333176 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9640/tid=9644] ppid=9638 vsize=1591080 CPUtime=871.44 cores=1,3,5,7
/proc/9640/task/9644/stat : 9644 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 58104 0 0 0 87111 33 0 0 25 0 5 0 498440182 1629265920 333176 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254332 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9640/tid=9645] ppid=9638 vsize=1591080 CPUtime=872.49 cores=1,3,5,7
/proc/9640/task/9645/stat : 9645 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 88441 0 0 0 87193 56 0 0 25 0 5 0 498440182 1629265920 333176 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3485.54
Current children cumulated vsize (KiB) 1591080

[startup+942.301 s]
/proc/loadavg: 4.75 6.87 7.59 5/176 9697
/proc/meminfo: memFree=18181556/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=1632128 CPUtime=3722.98 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) S 9638 9640 9362 0 -1 4202496 344846 0 0 0 372022 276 0 0 25 0 5 0 498439942 1671299072 344780 33554432000 4194304 4338100 140735553752512 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 408032 344780 198 36 0 405937 0
[pid=9640/tid=9642] ppid=9638 vsize=1632128 CPUtime=926.38 cores=1,3,5,7
/proc/9640/task/9642/stat : 9642 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 143842 0 0 0 92499 139 0 0 25 0 5 0 498440182 1671299072 344780 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9640/tid=9643] ppid=9638 vsize=1632128 CPUtime=931.39 cores=1,3,5,7
/proc/9640/task/9643/stat : 9643 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 52312 0 0 0 93096 43 0 0 25 0 5 0 498440182 1671299072 344780 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9640/tid=9644] ppid=9638 vsize=1632128 CPUtime=930.8 cores=1,3,5,7
/proc/9640/task/9644/stat : 9644 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 59268 0 0 0 93046 34 0 0 25 0 5 0 498440182 1671299072 344780 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9640/tid=9645] ppid=9638 vsize=1632128 CPUtime=931.97 cores=1,3,5,7
/proc/9640/task/9645/stat : 9645 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 88566 0 0 0 93139 58 0 0 18 0 5 0 498440182 1671299072 344780 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3722.98
Current children cumulated vsize (KiB) 1632128

[startup+1002.3 s]
/proc/loadavg: 4.32 6.36 7.37 5/176 9699
/proc/meminfo: memFree=18120548/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=1676432 CPUtime=3960.08 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) S 9638 9640 9362 0 -1 4202496 359819 0 0 0 395716 292 0 0 25 0 5 0 498439942 1716666368 359753 33554432000 4194304 4338100 140735553752512 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 419108 359753 198 36 0 417013 0
[pid=9640/tid=9642] ppid=9638 vsize=1676432 CPUtime=985.37 cores=1,3,5,7
/proc/9640/task/9642/stat : 9642 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 148870 0 0 0 98390 147 0 0 25 0 5 0 498440182 1716666368 359753 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254337 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9640/tid=9643] ppid=9638 vsize=1676432 CPUtime=990.92 cores=1,3,5,7
/proc/9640/task/9643/stat : 9643 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 58445 0 0 0 99045 47 0 0 25 0 5 0 498440182 1716666368 359753 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9640/tid=9644] ppid=9638 vsize=1676432 CPUtime=990.19 cores=1,3,5,7
/proc/9640/task/9644/stat : 9644 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 63067 0 0 0 98983 36 0 0 25 0 5 0 498440182 1716666368 359753 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9640/tid=9645] ppid=9638 vsize=1676432 CPUtime=991.18 cores=1,3,5,7
/proc/9640/task/9645/stat : 9645 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 88579 0 0 0 99058 60 0 0 25 0 5 0 498440182 1716666368 359753 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254239 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3960.08
Current children cumulated vsize (KiB) 1676432

[startup+1062.3 s]
/proc/loadavg: 4.16 5.95 7.16 5/176 9700
/proc/meminfo: memFree=18079612/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=1773504 CPUtime=4197.87 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) S 9638 9640 9362 0 -1 4202496 370336 0 0 0 419478 309 0 0 25 0 5 0 498439942 1816068096 370270 33554432000 4194304 4338100 140735553752512 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 443376 370270 198 36 0 441281 0
[pid=9640/tid=9642] ppid=9638 vsize=1773504 CPUtime=1044.73 cores=1,3,5,7
/proc/9640/task/9642/stat : 9642 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 153086 0 0 0 104319 154 0 0 25 0 5 0 498440182 1816068096 370270 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9640/tid=9643] ppid=9638 vsize=1773504 CPUtime=1050.51 cores=1,3,5,7
/proc/9640/task/9643/stat : 9643 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 63222 0 0 0 104999 52 0 0 25 0 5 0 498440182 1816068096 370270 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9640/tid=9644] ppid=9638 vsize=1773504 CPUtime=1049.52 cores=1,3,5,7
/proc/9640/task/9644/stat : 9644 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 64543 0 0 0 104913 39 0 0 23 0 5 0 498440182 1816068096 370270 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9640/tid=9645] ppid=9638 vsize=1773504 CPUtime=1050.67 cores=1,3,5,7
/proc/9640/task/9645/stat : 9645 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 88627 0 0 0 105005 62 0 0 23 0 5 0 498440182 1816068096 370270 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4197.87
Current children cumulated vsize (KiB) 1773504

[startup+1122.3 s]
/proc/loadavg: 4.09 5.60 6.96 5/176 9702
/proc/meminfo: memFree=17923856/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=1924032 CPUtime=4434.31 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) S 9638 9640 9362 0 -1 4202496 409332 0 0 0 443091 340 0 0 25 0 5 0 498439942 1970208768 409266 33554432000 4194304 4338100 140735553752512 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 481008 409266 198 36 0 478913 0
[pid=9640/tid=9642] ppid=9638 vsize=1924032 CPUtime=1103.52 cores=1,3,5,7
/proc/9640/task/9642/stat : 9642 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 167449 0 0 0 110186 166 0 0 25 0 5 0 498440182 1970208768 409266 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9640/tid=9643] ppid=9638 vsize=1924032 CPUtime=1110.06 cores=1,3,5,7
/proc/9640/task/9643/stat : 9643 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 86124 0 0 0 110942 64 0 0 25 0 5 0 498440182 1970208768 409266 33554432000 4194304 4338100 140735553752512 18446744073709551615 255106163942 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9640/tid=9644] ppid=9638 vsize=1924032 CPUtime=1108.36 cores=1,3,5,7
/proc/9640/task/9644/stat : 9644 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 65827 0 0 0 110795 41 0 0 25 0 5 0 498440182 1970208768 409266 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9640/tid=9645] ppid=9638 vsize=1924032 CPUtime=1109.94 cores=1,3,5,7
/proc/9640/task/9645/stat : 9645 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 89074 0 0 0 110928 66 0 0 25 0 5 0 498440182 1970208768 409266 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4434.31
Current children cumulated vsize (KiB) 1924032

[startup+1182.3 s]
/proc/loadavg: 4.03 5.31 6.77 5/176 9704
/proc/meminfo: memFree=17862464/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=1974720 CPUtime=4671.14 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) S 9638 9640 9362 0 -1 4202496 424319 0 0 0 466760 354 0 0 25 0 5 0 498439942 2022113280 424253 33554432000 4194304 4338100 140735553752512 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 493680 424253 198 36 0 491585 0
[pid=9640/tid=9642] ppid=9638 vsize=1974720 CPUtime=1162.56 cores=1,3,5,7
/proc/9640/task/9642/stat : 9642 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 170080 0 0 0 116085 171 0 0 25 0 5 0 498440182 2022113280 424253 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9640/tid=9643] ppid=9638 vsize=1974720 CPUtime=1169.3 cores=1,3,5,7
/proc/9640/task/9643/stat : 9643 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 96472 0 0 0 116859 71 0 0 25 0 5 0 498440182 2022113280 424253 33554432000 4194304 4338100 140735553752512 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9640/tid=9644] ppid=9638 vsize=1974720 CPUtime=1167.53 cores=1,3,5,7
/proc/9640/task/9644/stat : 9644 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 67785 0 0 0 116710 43 0 0 25 0 5 0 498440182 2022113280 424253 33554432000 4194304 4338100 140735553752512 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9640/tid=9645] ppid=9638 vsize=1974720 CPUtime=1169.32 cores=1,3,5,7
/proc/9640/task/9645/stat : 9645 (ttsth-5-4) R 9638 9640 9362 0 -1 4202560 89124 0 0 0 116864 68 0 0 25 0 5 0 498440182 2022113280 424253 33554432000 4194304 4338100 140735553752512 18446744073709551615 255106163728 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4671.14
Current children cumulated vsize (KiB) 1974720



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1215 s]
/proc/loadavg: 4.07 5.20 6.69 2/173 9704
/proc/meminfo: memFree=19496288/32951124 swapFree=67111524/67111528
[pid=9640] ppid=9638 vsize=0 CPUtime=4800.14 cores=1,3,5,7
/proc/9640/stat : 9640 (ttsth-5-4) Z 9638 9640 9362 0 -1 4203548 424991 0 0 0 479648 366 0 0 18 0 1 0 498439942 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/9640/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 4800.14
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 0
Real time (s): 1215.01
CPU time (s): 4800.15
CPU user time (s): 4796.49
CPU system time (s): 3.66844
CPU usage (%): 395.072
Max. virtual memory (cumulated for all children) (KiB): 1974848

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.49
system time used= 3.66844
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 424991
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 14593
involuntary context switches= 32699

runsolver used 1.9497 second user time and 4.51231 second system time

The end

Launcher Data 

Begin job on node144 at 2011-04-29 08:57:18
IDJOB=3274250
IDBENCH=83156
IDSOLVER=1654
FILE ID=node144/3274250-1304060238
RUNJOBID= node144-1304057438-9381
PBS_JOBID= 13170439
Free space on /tmp= 72412 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/spence/sgen/sgen3-n190-s29985700-unsat.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3274250-1304060238/watcher-3274250-1304060238 -o /tmp/evaluation-result-3274250-1304060238/solver-3274250-1304060238 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3274250-1304060238.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f2cf61fc47972fb7e89be65d1a83332e
RANDOM SEED=1700775285

node144.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.16
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      17962028 kB
Buffers:       2207032 kB
Cached:       10554888 kB
SwapCached:          4 kB
Active:        6277868 kB
Inactive:      8145652 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      17962028 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:              24 kB
Writeback:           0 kB
AnonPages:     1660388 kB
Mapped:          15316 kB
Slab:           499268 kB
PageTables:       7236 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  1875720 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72412 MiB
End job on node144 at 2011-04-29 09:17:35