Trace number 3266534

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4391.28 1300.35

General information on the benchmark

NameSAT02/handmade/zhang-lintao/
xor-chain/x1_80.shuffled.cnf
MD5SUM53a7d1541ad9c0dde902ec3a505ae8fc
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark0.001998
Satisfiable
(Un)Satisfiability was proved
Number of variables238
Number of clauses634
Sum of the clauses size1900
Maximum clause length3
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 22
Number of clauses of size 3632
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3266534-1304105233.cnf 
0.00/0.00	c Using SATTIMEOUT of 4800
0.00/0.00	c Using SATRAM of 15500
0.00/0.00	c Size of subset node = 32
0.00/0.00	c Variables = 238
0.00/0.00	c Clauses = 634
0.00/0.00	c Literals = 1900
0.00/0.00	c Reduced size:
0.00/0.00	c Variables = 238
0.00/0.00	c Clauses = 634
0.00/0.00	c Literals = 1900
0.00/0.00	c a2
0.00/0.00	c a1
4.00/4.00	c Independent sub-problems = 1
4.00/4.00	c Sub-problem with 238 variables and 634 clauses
4.00/4.00	c using permutation method 3
4.00/4.00	c Ordering finished 4.000 seconds
4.00/4.00	c mapped 4.000 seconds
4.00/4.00	c permuted 4.000 seconds
4.00/4.00	c about to sort
4.00/4.00	c sorted
4.00/4.01	c clauses now 634
4.00/4.01	c init finished 4.010 seconds
4.00/4.01	c tree built 4.010 seconds
4.00/4.01	c 
4.00/4.01	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
4.00/4.01	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
4.00/4.01	c ................................ | ..........                    |                                  4.0         0.0             
4.00/4.01	c ................................ | ..........                  1 |                                  4.0         0.2             
4.00/4.01	c ................................ | ..........                  2 |                                  4.0         0.5             
4.00/4.01	c ................................ | ..........                  3 |                                  4.0         0.7             
4.00/4.01	c ................................ | ..........                  4 |                                  4.0         1.0             
4.00/4.01	c ................................ | ..........                  5 |                                  4.0         1.2             
4.00/4.01	c ................................ | ..........               1000 | 136          8                   4.0       249.4         4282
95.89/28.73	c ................................ | ..........            4101000 | 1194247      16                 96.0     42718.8     21978883
191.64/55.99	c ................................ | ..........            7720000 | 2321894      16                192.0     40208.3     29212583
287.69/83.78	c ................................ | ..........           11185000 | 3391162      16                288.0     38835.5     34999668
384.44/112.51	c ................................ | ..........           14550000 | 4430810      16                384.5     37844.3     39974074
479.89/140.53	c ................................ | *.........           17855000 | 5475372      16                480.0     37196.4     44146652
575.82/168.85	c ................................ | *.........           21145000 | 6552644      16                576.0     36710.1     48488271
671.96/197.11	c ................................ | *.........           24432000 | 7622755      16                672.0     36357.1     52814866
768.01/226.50	c ................................ | *.........           27725000 | 8693299      16                768.0     36099.3     57664443
863.92/255.12	c ................................ | *.........           31072000 | 9789416      16                864.0     35963.0     62352460
959.89/283.75	c ................................ | **........           34404000 | 10885115     16                960.0     35837.1     67504529
1055.97/311.61	c ................................ | **........           37681000 | 11947400     16               1056.0     35682.8     72255811
1151.79/339.35	c ................................ | **........           41073000 | 13036640     16               1152.0     35653.3     77654681
1247.80/367.95	c ................................ | **........           44409000 | 14084922     16               1248.0     35583.8     83098931
1343.74/396.19	c ................................ | **........           47664000 | 15141637     16               1344.0     35463.8     87643445
1439.73/424.88	c ................................ | ***.......           50901000 | 16189733     16               1440.0     35347.9     92174894
1535.83/453.56	c ................................ | ***.......           54290000 | 17289248     16               1536.1     35343.4     98278668
1631.88/482.13	c ................................ | ***.......           57488000 | 18323409     16               1632.0     35225.5    102995316
1727.66/511.29	c ................................ | ***.......           60849000 | 19408272     16               1728.0     35213.5    108136492
1823.74/539.49	c ................................ | ***.......           64047000 | 20436089     16               1824.0     35113.5    112266887
1919.83/567.85	c ................................ | ****......           67341000 | 21525389     16               1920.0     35073.3    117505640
2016.01/596.01	c ................................ | ****......           70611000 | 22569352     16               2016.0     35024.4    122350877
2111.78/623.38	c ................................ | ****......           74020000 | 23664457     16               2112.1     35045.7    126624538
2207.83/651.46	c ................................ | ****......           77341000 | 24754373     16               2208.0     35027.5    131947552
2303.99/679.51	c ................................ | ****......           80571000 | 25797420     16               2304.0     34969.6    136855809
2399.69/708.18	c ................................ | *****.....           83960000 | 26869243     16               2400.0     34983.2    143543746
2495.71/735.77	c ................................ | *****.....           87243000 | 27934926     16               2496.0     34953.0    147381158
2591.99/763.90	c ................................ | *****.....           90492000 | 28980042     16               2592.0     34912.0    151481224
2687.89/792.24	c ................................ | *****.....           93922000 | 30028498     16               2688.0     34941.1    157938464
2783.96/820.73	c ................................ | *****.....           97041000 | 31060597     16               2784.1     34856.1    161839059
2879.63/848.99	c ................................ | ******....          100305000 | 32124323     16               2880.0     34828.1    166565128
2975.85/877.54	c ................................ | ******....          103481000 | 33157061     16               2976.0     34771.8    171165985
3071.73/906.28	c ................................ | ******....          106783000 | 34219072     16               3072.0     34760.0    175875412
3167.88/934.54	c ................................ | ******....          109845000 | 35213886     16               3168.0     34673.3    179022863
3263.81/963.35	c ................................ | ******....          113038000 | 36254328     16               3264.0     34631.7    183362676
3359.74/992.08	c ................................ | *******...          116211000 | 37288119     16               3360.0     34586.4    187487638
3455.62/1020.70	c ................................ | *******...          119349000 | 38315166     16               3456.0     34533.8    190973571
3551.80/1049.16	c ................................ | *******...          122616000 | 39381563     16               3552.0     34520.1    196205442
3647.99/1077.60	c ................................ | *******...          125776000 | 40399321     16               3648.0     34478.0    200045974
3743.86/1106.33	c ................................ | *******...          129033000 | 41454632     16               3744.0     34463.9    205730954
3839.75/1134.07	c ................................ | ********..          132211000 | 42495865     16               3840.0     34429.8    209316733
3935.94/1162.32	c ................................ | ********..          135512000 | 43554625     16               3936.0     34428.7    213709779
4031.77/1191.27	c ................................ | ********..          138751000 | 44608665     16               4032.0     34412.1    218112656
4128.01/1220.10	c ................................ | ********..          141973000 | 45653288     16               4128.0     34392.6    222771619
4223.93/1249.12	c ................................ | ********..          145202000 | 46706845     16               4224.0     34375.4    227662073
4319.97/1278.25	c ................................ | *********.          148401000 | 47730608     16               4320.1     34351.7    232379235

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3266534-1304105233/watcher-3266534-1304105233 -o /tmp/evaluation-result-3266534-1304105233/solver-3266534-1304105233 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3266534-1304105233.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.08 8.12 8.03 2/177 2680
/proc/meminfo: memFree=25620428/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=8840 CPUtime=0 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) R 2678 2680 2480 0 -1 4202496 313 0 0 0 0 0 0 0 25 0 1 0 502938739 9052160 255 33554432000 4194304 4338100 140735310036096 18446744073709551615 4245888 0 0 4096 0 0 0 0 17 0 0 0 0
/proc/2680/statm: 2210 255 163 36 0 115 0

[startup+0.0297629 s]
/proc/loadavg: 8.08 8.12 8.03 2/177 2680
/proc/meminfo: memFree=25620428/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=8840 CPUtime=0.02 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) R 2678 2680 2480 0 -1 4202496 313 0 0 0 2 0 0 0 25 0 1 0 502938739 9052160 255 33554432000 4194304 4338100 140735310036096 18446744073709551615 4304552 0 0 4096 0 0 0 0 17 0 0 0 0
/proc/2680/statm: 2210 255 163 36 0 115 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 8840

[startup+0.100776 s]
/proc/loadavg: 8.08 8.12 8.03 2/177 2680
/proc/meminfo: memFree=25620428/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=8840 CPUtime=0.1 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) R 2678 2680 2480 0 -1 4202496 323 0 0 0 10 0 0 0 25 0 1 0 502938739 9052160 265 33554432000 4194304 4338100 140735310036096 18446744073709551615 4245865 0 0 4096 0 0 0 0 17 0 0 0 0
/proc/2680/statm: 2210 265 173 36 0 115 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 8840

[startup+0.300715 s]
/proc/loadavg: 8.08 8.12 8.03 2/177 2680
/proc/meminfo: memFree=25620428/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=8840 CPUtime=0.3 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) R 2678 2680 2480 0 -1 4202496 323 0 0 0 30 0 0 0 25 0 1 0 502938739 9052160 265 33554432000 4194304 4338100 140735310036096 18446744073709551615 4245865 0 0 4096 0 0 0 0 17 0 0 0 0
/proc/2680/statm: 2210 265 173 36 0 115 0
Current children cumulated CPU time (s) 0.3
Current children cumulated vsize (KiB) 8840

[startup+0.70063 s]
/proc/loadavg: 8.08 8.12 8.03 2/177 2680
/proc/meminfo: memFree=25620428/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=8840 CPUtime=0.7 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) R 2678 2680 2480 0 -1 4202496 323 0 0 0 70 0 0 0 25 0 1 0 502938739 9052160 265 33554432000 4194304 4338100 140735310036096 18446744073709551615 4290907 0 0 4096 0 0 0 0 17 0 0 0 0
/proc/2680/statm: 2210 265 173 36 0 115 0
Current children cumulated CPU time (s) 0.7
Current children cumulated vsize (KiB) 8840

[startup+1.50046 s]
/proc/loadavg: 8.08 8.12 8.03 6/182 2685
/proc/meminfo: memFree=25585940/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=8840 CPUtime=1.5 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) R 2678 2680 2480 0 -1 4202496 323 0 0 0 150 0 0 0 25 0 1 0 502938739 9052160 265 33554432000 4194304 4338100 140735310036096 18446744073709551615 4245907 0 0 4096 0 0 0 0 17 0 0 0 0
/proc/2680/statm: 2210 265 173 36 0 115 0
Current children cumulated CPU time (s) 1.5
Current children cumulated vsize (KiB) 8840

[startup+3.10016 s]
/proc/loadavg: 7.83 8.07 8.02 6/182 2685
/proc/meminfo: memFree=25581228/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=8840 CPUtime=3.1 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) R 2678 2680 2480 0 -1 4202496 325 0 0 0 310 0 0 0 25 0 1 0 502938739 9052160 267 33554432000 4194304 4338100 140735310036096 18446744073709551615 4207022 0 0 4096 0 0 0 0 17 0 0 0 0
/proc/2680/statm: 2210 267 175 36 0 115 0
Current children cumulated CPU time (s) 3.1
Current children cumulated vsize (KiB) 8840

[startup+6.30054 s]
/proc/loadavg: 7.83 8.07 8.02 9/186 2689
/proc/meminfo: memFree=25416280/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=422904 CPUtime=13.03 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) S 2678 2680 2480 0 -1 4202496 59798 0 0 0 1293 10 0 0 25 0 5 0 502938739 433053696 59740 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/2680/statm: 105726 59740 198 36 0 103631 0
[pid=2680/tid=2686] ppid=2678 vsize=422904 CPUtime=2.24 cores=0,2,4,6
/proc/2680/task/2686/stat : 2686 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 14951 0 0 0 222 2 0 0 18 0 5 0 502939140 433053696 59741 33554432000 4194304 4338100 140735310036096 18446744073709551615 228094845968 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2680/tid=2687] ppid=2678 vsize=422904 CPUtime=2.26 cores=0,2,4,6
/proc/2680/task/2687/stat : 2687 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 14559 0 0 0 224 2 0 0 18 0 5 0 502939140 433053696 59741 33554432000 4194304 4338100 140735310036096 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=2680/tid=2688] ppid=2678 vsize=422904 CPUtime=2.25 cores=0,2,4,6
/proc/2680/task/2688/stat : 2688 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 14876 0 0 0 222 3 0 0 18 0 5 0 502939140 433053696 59741 33554432000 4194304 4338100 140735310036096 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2680/tid=2689] ppid=2678 vsize=422904 CPUtime=2.24 cores=0,2,4,6
/proc/2680/task/2689/stat : 2689 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 14403 0 0 0 222 2 0 0 18 0 5 0 502939140 433053696 59741 33554432000 4194304 4338100 140735310036096 18446744073709551615 228094846061 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 13.03
Current children cumulated vsize (KiB) 422904

[startup+12.7004 s]
/proc/loadavg: 7.94 8.08 8.02 10/186 2689
/proc/meminfo: memFree=25062120/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=761772 CPUtime=37.72 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) S 2678 2680 2480 0 -1 4202496 125938 0 0 0 3743 29 0 0 25 0 5 0 502938739 780054528 125860 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/2680/statm: 190443 125860 198 36 0 188348 0
[pid=2680/tid=2686] ppid=2678 vsize=761772 CPUtime=8.27 cores=0,2,4,6
/proc/2680/task/2686/stat : 2686 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 37202 0 0 0 819 8 0 0 18 0 5 0 502939140 780054528 125860 33554432000 4194304 4338100 140735310036096 18446744073709551615 228094846004 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2680/tid=2687] ppid=2678 vsize=761772 CPUtime=8.51 cores=0,2,4,6
/proc/2680/task/2687/stat : 2687 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 29734 0 0 0 845 6 0 0 18 0 5 0 502939140 780054528 125861 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107263750 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=2680/tid=2688] ppid=2678 vsize=761772 CPUtime=8.45 cores=0,2,4,6
/proc/2680/task/2688/stat : 2688 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 28919 0 0 0 838 7 0 0 18 0 5 0 502939140 780054528 125861 33554432000 4194304 4338100 140735310036096 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2680/tid=2689] ppid=2678 vsize=761772 CPUtime=8.46 cores=0,2,4,6
/proc/2680/task/2689/stat : 2689 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 29074 0 0 0 839 7 0 0 18 0 5 0 502939140 780054528 125861 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37.72
Current children cumulated vsize (KiB) 761772

[startup+25.501 s]
/proc/loadavg: 7.95 8.08 8.02 9/186 2690
/proc/meminfo: memFree=24810376/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=946612 CPUtime=84.77 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) S 2678 2680 2480 0 -1 4202496 172277 0 0 0 8429 48 0 0 25 0 5 0 502938739 969330688 172199 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/2680/statm: 236653 172199 198 36 0 234558 0
[pid=2680/tid=2686] ppid=2678 vsize=946612 CPUtime=19.38 cores=0,2,4,6
/proc/2680/task/2686/stat : 2686 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 66777 0 0 0 1923 15 0 0 16 0 5 0 502939140 969330688 172199 33554432000 4194304 4338100 140735310036096 18446744073709551615 4254471 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=2680/tid=2687] ppid=2678 vsize=946612 CPUtime=20.42 cores=0,2,4,6
/proc/2680/task/2687/stat : 2687 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 35649 0 0 0 2032 10 0 0 18 0 5 0 502939140 969330688 172199 33554432000 4194304 4338100 140735310036096 18446744073709551615 228094846177 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2680/tid=2688] ppid=2678 vsize=946612 CPUtime=20.45 cores=0,2,4,6
/proc/2680/task/2688/stat : 2688 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 35811 0 0 0 2034 11 0 0 18 0 5 0 502939140 969330688 172199 33554432000 4194304 4338100 140735310036096 18446744073709551615 4254816 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2680/tid=2689] ppid=2678 vsize=946612 CPUtime=20.47 cores=0,2,4,6
/proc/2680/task/2689/stat : 2689 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 33030 0 0 0 2037 10 0 0 17 0 5 0 502939140 969330688 172199 33554432000 4194304 4338100 140735310036096 18446744073709551615 4254024 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 84.77
Current children cumulated vsize (KiB) 946612

[startup+51.1003 s]
/proc/loadavg: 8.03 8.09 8.02 6/186 2690
/proc/meminfo: memFree=24474064/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=1218764 CPUtime=175.06 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) S 2678 2680 2480 0 -1 4202496 235718 0 0 0 17425 81 0 0 25 0 5 0 502938739 1248014336 235640 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/2680/statm: 304691 235640 198 36 0 302596 0
[pid=2680/tid=2686] ppid=2678 vsize=1218764 CPUtime=40.14 cores=0,2,4,6
/proc/2680/task/2686/stat : 2686 (ttsth-5-4) S 2678 2680 2480 0 -1 4202560 117950 0 0 0 3982 32 0 0 16 0 5 0 502939140 1248014336 235640 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107263750 0 0 4096 0 18446604453915228288 0 0 -1 4 0 0 0
[pid=2680/tid=2687] ppid=2678 vsize=1218764 CPUtime=43.67 cores=0,2,4,6
/proc/2680/task/2687/stat : 2687 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 41774 0 0 0 4351 16 0 0 18 0 5 0 502939140 1248014336 235640 33554432000 4194304 4338100 140735310036096 18446744073709551615 4235902 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2680/tid=2688] ppid=2678 vsize=1218764 CPUtime=43.44 cores=0,2,4,6
/proc/2680/task/2688/stat : 2688 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 37368 0 0 0 4329 15 0 0 16 0 5 0 502939140 1248014336 235640 33554432000 4194304 4338100 140735310036096 18446744073709551615 4235684 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2680/tid=2689] ppid=2678 vsize=1218764 CPUtime=43.75 cores=0,2,4,6
/proc/2680/task/2689/stat : 2689 (ttsth-5-4) S 2678 2680 2480 0 -1 4202560 37616 0 0 0 4359 16 0 0 18 0 5 0 502939140 1248014336 235640 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107263750 0 0 4096 0 18446604453915228288 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 175.06
Current children cumulated vsize (KiB) 1218764

[startup+102.307 s]
/proc/loadavg: 7.33 7.89 7.96 7/186 2692
/proc/meminfo: memFree=23955096/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=1635380 CPUtime=349.72 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) S 2678 2680 2480 0 -1 4202496 330144 0 0 0 34833 139 0 0 25 0 5 0 502938739 1674629120 330066 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/2680/statm: 408845 330066 198 36 0 406750 0
[pid=2680/tid=2686] ppid=2678 vsize=1635380 CPUtime=79.83 cores=0,2,4,6
/proc/2680/task/2686/stat : 2686 (ttsth-5-4) S 2678 2680 2480 0 -1 4202560 202186 0 0 0 7925 58 0 0 16 0 5 0 502939140 1674629120 330066 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107263750 0 0 4096 0 18446604453915228288 0 0 -1 4 0 0 0
[pid=2680/tid=2687] ppid=2678 vsize=1635380 CPUtime=88.88 cores=0,2,4,6
/proc/2680/task/2687/stat : 2687 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 42708 0 0 0 8861 27 0 0 17 0 5 0 502939140 1674629120 330066 33554432000 4194304 4338100 140735310036096 18446744073709551615 4234384 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2680/tid=2688] ppid=2678 vsize=1635380 CPUtime=88.37 cores=0,2,4,6
/proc/2680/task/2688/stat : 2688 (ttsth-5-4) S 2678 2680 2480 0 -1 4202560 39617 0 0 0 8813 24 0 0 16 0 5 0 502939140 1674629120 330066 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107263750 0 0 4096 0 18446604453915228288 0 0 -1 2 0 0 0
[pid=2680/tid=2689] ppid=2678 vsize=1635380 CPUtime=88.6 cores=0,2,4,6
/proc/2680/task/2689/stat : 2689 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 44623 0 0 0 8832 28 0 0 16 0 5 0 502939140 1674629120 330066 33554432000 4194304 4338100 140735310036096 18446744073709551615 4206915 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 349.72
Current children cumulated vsize (KiB) 1635380

[startup+162.301 s]
/proc/loadavg: 7.45 7.82 7.93 9/186 2693
/proc/meminfo: memFree=23496772/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=2019580 CPUtime=553.7 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) S 2678 2680 2480 0 -1 4202496 416188 0 0 0 55161 209 0 0 25 0 5 0 502938739 2068049920 416110 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/2680/statm: 504895 416110 198 36 0 502800 0
[pid=2680/tid=2686] ppid=2678 vsize=2019580 CPUtime=125.78 cores=0,2,4,6
/proc/2680/task/2686/stat : 2686 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 276672 0 0 0 12490 88 0 0 16 0 5 0 502939140 2068049920 416110 33554432000 4194304 4338100 140735310036096 18446744073709551615 4253966 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2680/tid=2687] ppid=2678 vsize=2019580 CPUtime=141.7 cores=0,2,4,6
/proc/2680/task/2687/stat : 2687 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 43183 0 0 0 14131 39 0 0 16 0 5 0 502939140 2068049920 416110 33554432000 4194304 4338100 140735310036096 18446744073709551615 228094846389 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2680/tid=2688] ppid=2678 vsize=2019580 CPUtime=141.04 cores=0,2,4,6
/proc/2680/task/2688/stat : 2688 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 43195 0 0 0 14068 36 0 0 16 0 5 0 502939140 2068049920 416110 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107263750 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2680/tid=2689] ppid=2678 vsize=2019580 CPUtime=141.12 cores=0,2,4,6
/proc/2680/task/2689/stat : 2689 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 52128 0 0 0 14069 43 0 0 16 0 5 0 502939140 2068049920 416110 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107265263 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 553.7
Current children cumulated vsize (KiB) 2019580

[startup+222.301 s]
/proc/loadavg: 7.66 7.82 7.92 7/186 2695
/proc/meminfo: memFree=23093748/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=2335804 CPUtime=754.01 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) S 2678 2680 2480 0 -1 4202496 498309 0 0 0 75121 280 0 0 25 0 5 0 502938739 2391863296 498231 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/2680/statm: 583951 498231 198 36 0 581856 0
[pid=2680/tid=2686] ppid=2678 vsize=2335804 CPUtime=171.23 cores=0,2,4,6
/proc/2680/task/2686/stat : 2686 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 345281 0 0 0 17007 116 0 0 16 0 5 0 502939140 2391863296 498231 33554432000 4194304 4338100 140735310036096 18446744073709551615 228095147550 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2680/tid=2687] ppid=2678 vsize=2335804 CPUtime=193.38 cores=0,2,4,6
/proc/2680/task/2687/stat : 2687 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 46743 0 0 0 19284 54 0 0 16 0 5 0 502939140 2391863296 498231 33554432000 4194304 4338100 140735310036096 18446744073709551615 228094846107 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2680/tid=2688] ppid=2678 vsize=2335804 CPUtime=192.28 cores=0,2,4,6
/proc/2680/task/2688/stat : 2688 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 45915 0 0 0 19178 50 0 0 16 0 5 0 502939140 2391863296 498231 33554432000 4194304 4338100 140735310036096 18446744073709551615 228094846690 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2680/tid=2689] ppid=2678 vsize=2335804 CPUtime=193.08 cores=0,2,4,6
/proc/2680/task/2689/stat : 2689 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 59360 0 0 0 19250 58 0 0 16 0 5 0 502939140 2391863296 498231 33554432000 4194304 4338100 140735310036096 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 754.01
Current children cumulated vsize (KiB) 2335804

################
# More data... #
################

/proc/2680/task/2689/stat : 2689 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 125987 0 0 0 71484 208 0 0 16 0 5 0 502939140 6810071040 1508371 33554432000 4194304 4338100 140735310036096 18446744073709551615 228094846094 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2789.28
Current children cumulated vsize (KiB) 6650460

[startup+882.301 s]
/proc/loadavg: 7.72 7.63 7.73 9/186 2713
/proc/meminfo: memFree=18087132/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=7278428 CPUtime=2991.9 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) S 2678 2680 2480 0 -1 4202496 1603770 0 0 0 298113 1077 0 0 25 0 5 0 502938739 7453110272 1603692 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/2680/statm: 1819607 1603692 198 36 0 1817512 0
[pid=2680/tid=2686] ppid=2678 vsize=7278428 CPUtime=681.89 cores=0,2,4,6
/proc/2680/task/2686/stat : 2686 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 1243587 0 0 0 67770 419 0 0 16 0 5 0 502939140 7453110272 1603692 33554432000 4194304 4338100 140735310036096 18446744073709551615 4235651 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=2680/tid=2687] ppid=2678 vsize=7278428 CPUtime=773.72 cores=0,2,4,6
/proc/2680/task/2687/stat : 2687 (ttsth-5-4) S 2678 2680 2480 0 -1 4202560 105740 0 0 0 77163 209 0 0 17 0 5 0 502939140 7453110272 1603692 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107263750 0 0 4096 0 18446604453915228288 0 0 -1 2 0 0 0
[pid=2680/tid=2688] ppid=2678 vsize=7278428 CPUtime=764.24 cores=0,2,4,6
/proc/2680/task/2688/stat : 2688 (ttsth-5-4) S 2678 2680 2480 0 -1 4202560 120613 0 0 0 76199 225 0 0 16 0 5 0 502939140 7453110272 1603692 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107263750 0 0 4096 0 18446604453915228288 0 0 -1 6 0 0 0
[pid=2680/tid=2689] ppid=2678 vsize=7278428 CPUtime=768.02 cores=0,2,4,6
/proc/2680/task/2689/stat : 2689 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 132820 0 0 0 76579 223 0 0 16 0 5 0 502939140 7453110272 1603692 33554432000 4194304 4338100 140735310036096 18446744073709551615 4235651 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2991.9
Current children cumulated vsize (KiB) 7278428

[startup+942.301 s]
/proc/loadavg: 7.44 7.59 7.71 10/186 2714
/proc/meminfo: memFree=17717832/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=7541180 CPUtime=3194.23 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) S 2678 2680 2480 0 -1 4202496 1684386 0 0 0 318281 1142 0 0 25 0 5 0 502938739 7722168320 1684308 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/2680/statm: 1885295 1684308 198 36 0 1883200 0
[pid=2680/tid=2686] ppid=2678 vsize=7541180 CPUtime=728.87 cores=0,2,4,6
/proc/2680/task/2686/stat : 2686 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 1309265 0 0 0 72446 441 0 0 16 0 5 0 502939140 7722168320 1684308 33554432000 4194304 4338100 140735310036096 18446744073709551615 4235651 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2680/tid=2687] ppid=2678 vsize=7541180 CPUtime=825.97 cores=0,2,4,6
/proc/2680/task/2687/stat : 2687 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 108577 0 0 0 82377 220 0 0 16 0 5 0 502939140 7722168320 1684308 33554432000 4194304 4338100 140735310036096 18446744073709551615 4235651 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2680/tid=2688] ppid=2678 vsize=7541180 CPUtime=815.4 cores=0,2,4,6
/proc/2680/task/2688/stat : 2688 (ttsth-5-4) S 2678 2680 2480 0 -1 4202560 129155 0 0 0 81297 243 0 0 16 0 5 0 502939140 7722168320 1684308 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107263750 0 0 4096 0 18446604453915228288 0 0 -1 4 0 0 0
[pid=2680/tid=2689] ppid=2678 vsize=7541180 CPUtime=819.95 cores=0,2,4,6
/proc/2680/task/2689/stat : 2689 (ttsth-5-4) S 2678 2680 2480 0 -1 4202560 136379 0 0 0 81758 237 0 0 16 0 5 0 502939140 7722168320 1684308 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107263750 0 0 4096 0 18446604453915228288 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3194.23
Current children cumulated vsize (KiB) 7541180

[startup+1002.3 s]
/proc/loadavg: 7.35 7.52 7.67 8/186 2716
/proc/meminfo: memFree=17361184/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=7810188 CPUtime=3393.83 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) S 2678 2680 2480 0 -1 4202496 1763285 0 0 0 338178 1205 0 0 25 0 5 0 502938739 7997632512 1763207 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/2680/statm: 1952547 1763207 198 36 0 1950452 0
[pid=2680/tid=2686] ppid=2678 vsize=7810188 CPUtime=774.22 cores=0,2,4,6
/proc/2680/task/2686/stat : 2686 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 1376460 0 0 0 76957 465 0 0 16 0 5 0 502939140 7997632512 1763207 33554432000 4194304 4338100 140735310036096 18446744073709551615 4236061 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2680/tid=2687] ppid=2678 vsize=7810188 CPUtime=877.01 cores=0,2,4,6
/proc/2680/task/2687/stat : 2687 (ttsth-5-4) S 2678 2680 2480 0 -1 4202560 109791 0 0 0 87468 233 0 0 16 0 5 0 502939140 7997632512 1763207 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107263750 0 0 4096 0 18446604453915228288 0 0 -1 4 0 0 0
[pid=2680/tid=2688] ppid=2678 vsize=7810188 CPUtime=867.23 cores=0,2,4,6
/proc/2680/task/2688/stat : 2688 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 134976 0 0 0 86466 257 0 0 15 0 5 0 502939140 7997632512 1763207 33554432000 4194304 4338100 140735310036096 18446744073709551615 4235651 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2680/tid=2689] ppid=2678 vsize=7810188 CPUtime=871.32 cores=0,2,4,6
/proc/2680/task/2689/stat : 2689 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 141048 0 0 0 86884 248 0 0 16 0 5 0 502939140 7997632512 1763207 33554432000 4194304 4338100 140735310036096 18446744073709551615 4235847 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3393.83
Current children cumulated vsize (KiB) 7810188

[startup+1062.3 s]
/proc/loadavg: 7.14 7.44 7.64 10/186 2717
/proc/meminfo: memFree=16973036/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=8088580 CPUtime=3596.64 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) S 2678 2680 2480 0 -1 4202496 1854178 0 0 0 358393 1271 0 0 25 0 5 0 502938739 8282705920 1854100 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/2680/statm: 2022145 1854100 198 36 0 2020050 0
[pid=2680/tid=2686] ppid=2678 vsize=8088580 CPUtime=820.5 cores=0,2,4,6
/proc/2680/task/2686/stat : 2686 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 1446010 0 0 0 81560 490 0 0 16 0 5 0 502939140 8282705920 1854100 33554432000 4194304 4338100 140735310036096 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2680/tid=2687] ppid=2678 vsize=8088580 CPUtime=929.7 cores=0,2,4,6
/proc/2680/task/2687/stat : 2687 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 115317 0 0 0 92724 246 0 0 16 0 5 0 502939140 8282705920 1854100 33554432000 4194304 4338100 140735310036096 18446744073709551615 228094845978 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=2680/tid=2688] ppid=2678 vsize=8088580 CPUtime=919.06 cores=0,2,4,6
/proc/2680/task/2688/stat : 2688 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 142171 0 0 0 91634 272 0 0 16 0 5 0 502939140 8282705920 1854100 33554432000 4194304 4338100 140735310036096 18446744073709551615 228094846061 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2680/tid=2689] ppid=2678 vsize=8088580 CPUtime=923.34 cores=0,2,4,6
/proc/2680/task/2689/stat : 2689 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 149670 0 0 0 92073 261 0 0 16 0 5 0 502939140 8282705920 1854100 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107265263 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3596.64
Current children cumulated vsize (KiB) 8088580

[startup+1122.3 s]
/proc/loadavg: 7.16 7.40 7.61 7/186 2719
/proc/meminfo: memFree=16518672/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=8741572 CPUtime=3798.64 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) S 2678 2680 2480 0 -1 4202496 1956499 0 0 0 378519 1345 0 0 25 0 5 0 502938739 8951369728 1956421 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/2680/statm: 2185393 1956421 198 36 0 2183298 0
[pid=2680/tid=2686] ppid=2678 vsize=8741572 CPUtime=866.77 cores=0,2,4,6
/proc/2680/task/2686/stat : 2686 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 1527902 0 0 0 86160 517 0 0 16 0 5 0 502939140 8951369728 1956421 33554432000 4194304 4338100 140735310036096 18446744073709551615 228095147510 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2680/tid=2687] ppid=2678 vsize=8741572 CPUtime=982.53 cores=0,2,4,6
/proc/2680/task/2687/stat : 2687 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 120356 0 0 0 97986 267 0 0 18 0 5 0 502939140 8951369728 1956421 33554432000 4194304 4338100 140735310036096 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=2680/tid=2688] ppid=2678 vsize=8741572 CPUtime=970.64 cores=0,2,4,6
/proc/2680/task/2688/stat : 2688 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 151109 0 0 0 96779 285 0 0 16 0 5 0 502939140 8951369728 1956421 33554432000 4194304 4338100 140735310036096 18446744073709551615 228094846221 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2680/tid=2689] ppid=2678 vsize=8741572 CPUtime=974.67 cores=0,2,4,6
/proc/2680/task/2689/stat : 2689 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 156122 0 0 0 97192 275 0 0 16 0 5 0 502939140 8951369728 1956421 33554432000 4194304 4338100 140735310036096 18446744073709551615 228094846075 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3798.64
Current children cumulated vsize (KiB) 8741572

[startup+1182.3 s]
/proc/loadavg: 7.50 7.44 7.61 9/186 2721
/proc/meminfo: memFree=16139216/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=8985556 CPUtime=4002.85 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) S 2678 2680 2480 0 -1 4202496 2031032 0 0 0 398870 1415 0 0 25 0 5 0 502938739 9201209344 2030954 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/2680/statm: 2246389 2030954 198 36 0 2244294 0
[pid=2680/tid=2686] ppid=2678 vsize=8985556 CPUtime=913.61 cores=0,2,4,6
/proc/2680/task/2686/stat : 2686 (ttsth-5-4) S 2678 2680 2480 0 -1 4202560 1588464 0 0 0 90818 543 0 0 16 0 5 0 502939140 9201209344 2030954 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107276660 0 0 4096 0 18446604453915228288 0 0 -1 2 0 0 0
[pid=2680/tid=2687] ppid=2678 vsize=8985556 CPUtime=1035.29 cores=0,2,4,6
/proc/2680/task/2687/stat : 2687 (ttsth-5-4) S 2678 2680 2480 0 -1 4202560 123932 0 0 0 103248 281 0 0 16 0 5 0 502939140 9201209344 2030954 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107276660 0 0 4096 0 18446604453915228288 0 0 -1 0 0 0 0
[pid=2680/tid=2688] ppid=2678 vsize=8985556 CPUtime=1022.82 cores=0,2,4,6
/proc/2680/task/2688/stat : 2688 (ttsth-5-4) S 2678 2680 2480 0 -1 4202560 158794 0 0 0 101983 299 0 0 16 0 5 0 502939140 9201209344 2030954 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107276660 0 0 4096 0 18446604453915228288 0 0 -1 4 0 0 0
[pid=2680/tid=2689] ppid=2678 vsize=8985556 CPUtime=1027.09 cores=0,2,4,6
/proc/2680/task/2689/stat : 2689 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 158832 0 0 0 102418 291 0 0 16 0 5 0 502939140 9201209344 2030954 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107263694 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4002.85
Current children cumulated vsize (KiB) 8985556

[startup+1242.3 s]
/proc/loadavg: 7.46 7.44 7.60 6/186 2759
/proc/meminfo: memFree=15824512/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=9323380 CPUtime=4200.98 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) S 2678 2680 2480 0 -1 4202496 2131944 0 0 0 418612 1486 0 0 25 0 5 0 502938739 9547141120 2131866 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/2680/statm: 2330845 2131866 198 36 0 2328750 0
[pid=2680/tid=2686] ppid=2678 vsize=9323380 CPUtime=958.1 cores=0,2,4,6
/proc/2680/task/2686/stat : 2686 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 1673144 0 0 0 95239 571 0 0 16 0 5 0 502939140 9547141120 2131866 33554432000 4194304 4338100 140735310036096 18446744073709551615 4256960 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=2680/tid=2687] ppid=2678 vsize=9323380 CPUtime=1088.16 cores=0,2,4,6
/proc/2680/task/2687/stat : 2687 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 130417 0 0 0 108519 297 0 0 16 0 5 0 502939140 9547141120 2131866 33554432000 4194304 4338100 140735310036096 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2680/tid=2688] ppid=2678 vsize=9323380 CPUtime=1073.06 cores=0,2,4,6
/proc/2680/task/2688/stat : 2688 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 161698 0 0 0 106993 313 0 0 16 0 5 0 502939140 9547141120 2131866 33554432000 4194304 4338100 140735310036096 18446744073709551615 4242703 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2680/tid=2689] ppid=2678 vsize=9323380 CPUtime=1077.62 cores=0,2,4,6
/proc/2680/task/2689/stat : 2689 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 165675 0 0 0 107458 304 0 0 16 0 5 0 502939140 9547141120 2131866 33554432000 4194304 4338100 140735310036096 18446744073709551615 4206915 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4200.98
Current children cumulated vsize (KiB) 9323380



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 7.48 7.44 7.58 9/186 2761
/proc/meminfo: memFree=14354084/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=9626796 CPUtime=4390.9 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) S 2678 2680 2480 0 -1 4202496 2224236 0 0 0 437534 1556 0 0 25 0 5 0 502938739 9857839104 2224158 33554432000 4194304 4338100 140735310036096 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/2680/statm: 2406699 2224158 198 36 0 2404604 0
[pid=2680/tid=2686] ppid=2678 vsize=9626796 CPUtime=1000.65 cores=0,2,4,6
/proc/2680/task/2686/stat : 2686 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 1748717 0 0 0 99469 596 0 0 16 0 5 0 502939140 9857839104 2224158 33554432000 4194304 4338100 140735310036096 18446744073709551615 228094846398 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2680/tid=2687] ppid=2678 vsize=9626796 CPUtime=1137.98 cores=0,2,4,6
/proc/2680/task/2687/stat : 2687 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 134372 0 0 0 113485 313 0 0 16 0 5 0 502939140 9857839104 2224158 33554432000 4194304 4338100 140735310036096 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2680/tid=2688] ppid=2678 vsize=9626796 CPUtime=1121.79 cores=0,2,4,6
/proc/2680/task/2688/stat : 2688 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 169245 0 0 0 111850 329 0 0 16 0 5 0 502939140 9857839104 2224158 33554432000 4194304 4338100 140735310036096 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2680/tid=2689] ppid=2678 vsize=9626796 CPUtime=1126.45 cores=0,2,4,6
/proc/2680/task/2689/stat : 2689 (ttsth-5-4) R 2678 2680 2480 0 -1 4202560 170892 0 0 0 112328 317 0 0 16 0 5 0 502939140 9857839104 2224158 33554432000 4194304 4338100 140735310036096 18446744073709551615 4242623 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4390.9
Current children cumulated vsize (KiB) 9626796

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1300.21 s]
/proc/loadavg: 7.48 7.44 7.58 6/184 2762
/proc/meminfo: memFree=16681332/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=0 CPUtime=4391.13 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) Z 2678 2680 2480 0 -1 4203548 2224236 0 0 0 437537 1576 0 0 18 0 2 0 502938739 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/2680/statm: 0 0 0 0 0 0 0
[pid=2680/tid=2686] ppid=2678 vsize=0 CPUtime=1000.86 cores=0,2,4,6
/proc/2680/task/2686/stat : 2686 (ttsth-5-4) R 2678 2680 2480 0 -1 4203588 1748717 0 0 0 99470 616 0 0 16 0 2 0 502939140 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4391.13
Current children cumulated vsize (KiB) 0

[startup+1300.3 s]
/proc/loadavg: 7.48 7.44 7.58 6/184 2762
/proc/meminfo: memFree=16681332/32951124 swapFree=67111524/67111528
[pid=2680] ppid=2678 vsize=0 CPUtime=4391.22 cores=0,2,4,6
/proc/2680/stat : 2680 (ttsth-5-4) Z 2678 2680 2480 0 -1 4203548 2224236 0 0 0 437537 1585 0 0 18 0 2 0 502938739 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/2680/statm: 0 0 0 0 0 0 0
[pid=2680/tid=2686] ppid=2678 vsize=0 CPUtime=1000.95 cores=0,2,4,6
/proc/2680/task/2686/stat : 2686 (ttsth-5-4) R 2678 2680 2480 0 -1 4203588 1748717 0 0 0 99470 625 0 0 16 0 2 0 502939140 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4391.22
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1300.35
CPU time (s): 4391.28
CPU user time (s): 4375.37
CPU system time (s): 15.9026
CPU usage (%): 337.7
Max. virtual memory (cumulated for all children) (KiB): 9626796

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4375.37
system time used= 15.9026
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2224236
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2731039
involuntary context switches= 20744

runsolver used 1.81372 second user time and 4.41133 second system time

The end

Launcher Data 

Begin job on node142 at 2011-04-29 21:27:13
IDJOB=3266534
IDBENCH=87811
IDSOLVER=1654
FILE ID=node142/3266534-1304105233
RUNJOBID= node142-1304104028-2498
PBS_JOBID= 13170645
Free space on /tmp= 71316 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT02/handmade/zhang-lintao/xor-chain/x1_80.shuffled.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3266534-1304105233/watcher-3266534-1304105233 -o /tmp/evaluation-result-3266534-1304105233/solver-3266534-1304105233 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3266534-1304105233.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 53a7d1541ad9c0dde902ec3a505ae8fc
RANDOM SEED=76911704

node142.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5131.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25620452 kB
Buffers:       1682728 kB
Cached:        5038448 kB
SwapCached:          4 kB
Active:         877636 kB
Inactive:      5911452 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25620452 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:              32 kB
Writeback:           0 kB
AnonPages:       67452 kB
Mapped:          15088 kB
Slab:           478124 kB
PageTables:       4288 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   209468 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71312 MiB
End job on node142 at 2011-04-29 21:48:55