Trace number 3262090

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.54 1202.37

General information on the benchmark

Namecrafted/kullmann/VanDerWaerden/VanderWaerden_pd_3k/
VanDerWaerden_pd_2-3-21_404.cnf
MD5SUM213e4c0c58f043e67e86cf993698317a
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark8.22675
Satisfiable
(Un)Satisfiability was proved
Number of variables202
Number of clauses22023
Sum of the clauses size98860
Maximum clause length21
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 267
Number of clauses of size 320125
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 51831

Solver Data 

0.00/0.02	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262090-1304091097.cnf 
0.00/0.02	c Using SATTIMEOUT of 4800
0.00/0.02	c Using SATRAM of 15500
0.00/0.02	c Size of subset node = 32
0.00/0.02	c Variables = 202
0.00/0.02	c Clauses = 22023
0.00/0.02	c Literals = 98860
0.00/0.02	c Reduced size:
0.00/0.02	c Variables = 202
0.00/0.02	c Clauses = 22023
0.00/0.02	c Literals = 98860
0.00/0.02	c a2
0.09/0.13	c a1
0.29/0.32	c Independent sub-problems = 1
0.29/0.32	c Sub-problem with 202 variables and 22023 clauses
0.29/0.32	c using permutation method 4
0.29/0.32	c count = 1
0.29/0.32	c Ordering finished 0.310 seconds
0.29/0.32	c mapped 0.310 seconds
0.29/0.32	c permuted 0.310 seconds
0.29/0.32	c about to sort
0.29/0.32	c sorted
0.29/0.39	c clauses now 22023
0.29/0.39	c init finished 0.320 seconds
0.29/0.39	c tree built 0.380 seconds
0.29/0.39	c 
0.29/0.39	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.29/0.39	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.29/0.39	c ................................ | ..........                    |                                  0.4         0.0             
0.29/0.39	c ................................ | ..........                  1 |                                  0.4         2.6             
0.29/0.39	c ................................ | ..........                  2 |                                  0.4         5.3             
0.29/0.39	c ................................ | ..........                  3 |                                  0.4         7.9             
0.29/0.39	c ................................ | ..........                  4 |                                  0.4        10.5             
0.29/0.39	c ................................ | ..........                  5 |                                  0.4        13.2             
0.39/0.50	c ................................ | ..........               1000 | 6            402                 0.5      2127.7        32551
95.86/24.35	c ................................ | ..........             441000 | 37952        146854             96.1      4590.4     29012664
192.53/48.50	c ................................ | ..........             649000 | 60598        233963            192.5      3370.9     45197351
288.41/72.51	c ................................ | ..........             801000 | 76450        308716            288.5      2776.8     59755294
384.26/96.54	c ................................ | ..........             942000 | 93766        366066            384.4      2450.4     71862644
480.13/120.51	c ................................ | *.........            1084000 | 107561       412523            480.2      2257.5     82195933
576.39/144.68	c ................................ | *.........            1196000 | 120616       457569            576.7      2073.7     91960631
671.86/168.54	c ................................ | *.........            1306000 | 133490       498902            672.0      1943.3    100717374
768.52/192.71	c ................................ | *.........            1403000 | 144207       545089            768.6      1825.4    109661023
864.00/216.67	c ................................ | *.........            1489000 | 153356       590550            864.3      1722.8    116832693
960.27/240.80	c ................................ | **........            1565000 | 162846       631261            960.7      1629.1    124216489
1055.73/264.68	c ................................ | **........            1634000 | 170799       666456           1056.0      1547.3    131153696
1152.80/288.90	c ................................ | **........            1707000 | 177038       694063           1152.8      1480.7    137470426
1248.26/312.84	c ................................ | **........            1772000 | 182654       722470           1248.4      1419.4    144105591
1344.14/336.80	c ................................ | **........            1828000 | 188161       753456           1344.1      1360.0    150680911
1441.20/361.16	c ................................ | ***.......            1894000 | 196112       782603           1441.4      1314.0    156160607
1536.70/385.02	c ................................ | ***.......            1958000 | 204009       807270           1536.8      1274.1    162314617
1632.14/408.99	c ................................ | ***.......            2013000 | 210585       834774           1632.5      1233.1    167270851
1728.81/433.17	c ................................ | ***.......            2068000 | 217091       862955           1729.1      1196.0    172381999
1823.88/456.94	c ................................ | ***.......            2122000 | 222556       889306           1824.1      1163.3    178093060
1921.35/481.34	c ................................ | ****......            2170000 | 228010       921194           1921.5      1129.3    183055877
2017.62/505.45	c ................................ | ****......            2220000 | 234214       944604           2017.8      1100.2    189351603
2113.49/529.46	c ................................ | ****......            2269000 | 240299       968036           2113.7      1073.5    194607287
2208.16/553.17	c ................................ | ****......            2313000 | 245704       988879           2208.4      1047.3    199483135
2304.43/577.26	c ................................ | ****......            2358000 | 251251       1011379          2304.7      1023.1    204224723
2402.32/601.71	c ................................ | *****.....            2399000 | 256415       1034579          2402.3       998.6    208736223
2497.76/625.67	c ................................ | *****.....            2441000 | 262036       1058068          2498.1       977.2    212684377
2592.03/649.26	c ................................ | *****.....            2482000 | 267170       1081322          2592.3       957.5    216371579
2687.92/673.22	c ................................ | *****.....            2524000 | 272456       1100213          2688.0       939.0    220737440
2785.37/697.63	c ................................ | *****.....            2564000 | 277410       1120424          2785.5       920.5    224415144
2882.04/721.88	c ................................ | ******....            2603000 | 282572       1139375          2882.4       903.1    228564790
2975.88/745.38	c ................................ | ******....            2643000 | 287618       1155794          2976.2       888.0    230368764
3073.34/769.73	c ................................ | ******....            2683000 | 292663       1172763          3073.5       873.0    231283446
3170.01/793.96	c ................................ | ******....            2721000 | 297292       1189172          3170.3       858.3    233974748
3263.87/817.49	c ................................ | ******....            2757000 | 301825       1206840          3264.3       844.6    237371471
3360.89/841.76	c ................................ | *******...            2790000 | 306026       1220872          3361.2       830.1    240972030
3455.97/865.58	c ................................ | *******...            2827000 | 310564       1236542          3456.3       817.9    240972030
3553.45/889.99	c ................................ | *******...            2866000 | 315344       1255415          3553.8       806.5    240972030
3648.51/913.76	c ................................ | *******...            2902000 | 319876       1271864          3648.8       795.3    241261596
3743.97/937.63	c ................................ | *******...            2940000 | 324649       1286003          3744.1       785.2    243227492
3842.64/962.39	c ................................ | ********..            2977000 | 329291       1300866          3843.0       774.7    245719182
3937.70/986.12	c ................................ | ********..            3008000 | 333697       1315587          3937.8       763.9    247431733
4032.78/1009.97	c ................................ | ********..            3040000 | 337468       1328634          4033.1       753.8    247507349
4130.65/1034.49	c ................................ | ********..            3073000 | 341661       1343127          4131.0       743.9    248425479
4224.13/1057.85	c ................................ | ********..            3105000 | 345550       1357339          4224.4       735.0    249117938
4322.00/1082.40	c ................................ | *********.            3137000 | 349324       1369528          4322.4       725.8    250921020
4417.85/1106.35	c ................................ | *********.            3170000 | 352287       1387261          4418.1       717.5    253139893
4512.92/1130.14	c ................................ | *********.            3210000 | 355740       1405217          4513.1       711.3    255666212
4609.59/1154.33	c ................................ | *********.            3249000 | 359918       1422534          4609.7       704.8    258344460
4704.68/1178.16	c ................................ | *********.            3280000 | 363443       1444112          4704.9       697.1    262470174

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3262090-1304091097/watcher-3262090-1304091097 -o /tmp/evaluation-result-3262090-1304091097/solver-3262090-1304091097 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262090-1304091097.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.33 6.01 7.01 2/176 1713
/proc/meminfo: memFree=30925892/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=9112 CPUtime=0 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) R 1708 1713 1676 0 -1 4202496 357 0 0 0 0 0 0 0 22 0 1 0 501525631 9330688 299 33554432000 4194304 4338100 140737150698064 18446744073709551615 4223897 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/1713/statm: 2278 300 146 36 0 183 0

[startup+0.0976301 s]
/proc/loadavg: 3.33 6.01 7.01 2/176 1713
/proc/meminfo: memFree=30925892/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=24332 CPUtime=0.09 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) R 1708 1713 1676 0 -1 4202496 3481 0 0 0 9 0 0 0 22 0 1 0 501525631 24915968 3346 33554432000 4194304 4338100 140737150698064 18446744073709551615 4206978 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/1713/statm: 6083 3346 363 36 0 3988 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 24332

[startup+0.100626 s]
/proc/loadavg: 3.33 6.01 7.01 2/176 1713
/proc/meminfo: memFree=30925892/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=24332 CPUtime=0.09 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) R 1708 1713 1676 0 -1 4202496 3481 0 0 0 9 0 0 0 22 0 1 0 501525631 24915968 3346 33554432000 4194304 4338100 140737150698064 18446744073709551615 220815631554 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/1713/statm: 6083 3346 363 36 0 3988 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 24332

[startup+0.300569 s]
/proc/loadavg: 3.33 6.01 7.01 2/176 1713
/proc/meminfo: memFree=30925892/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=14464 CPUtime=0.29 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) R 1708 1713 1676 0 -1 4202496 4161 0 0 0 29 0 0 0 23 0 1 0 501525631 14811136 1634 33554432000 4194304 4338100 140737150698064 18446744073709551615 4297518 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/1713/statm: 3616 1634 156 36 0 1521 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 14464

[startup+0.700429 s]
/proc/loadavg: 3.33 6.01 7.01 2/176 1713
/proc/meminfo: memFree=30925892/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=164068 CPUtime=1.59 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) S 1708 1713 1676 0 -1 4202496 13942 0 0 0 158 1 0 0 24 0 5 0 501525631 168005632 11327 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 41017 11328 176 36 0 38922 0
Current children cumulated CPU time (s) 1.59
Current children cumulated vsize (KiB) 164068

[startup+1.50028 s]
/proc/loadavg: 3.33 6.01 7.01 11/186 1723
/proc/meminfo: memFree=30798480/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=283180 CPUtime=4.78 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) S 1708 1713 1676 0 -1 4202496 26640 0 0 0 475 3 0 0 24 0 5 0 501525631 289976320 24025 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 70795 24026 176 36 0 68700 0
[pid=1713/tid=1720] ppid=1708 vsize=283180 CPUtime=1.09 cores=0,2,4,6
/proc/1713/task/1720/stat : 1720 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 4862 0 0 0 109 0 0 0 19 0 5 0 501525671 289976320 24026 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828049190 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=1713/tid=1721] ppid=1708 vsize=283180 CPUtime=1.09 cores=0,2,4,6
/proc/1713/task/1721/stat : 1721 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 4951 0 0 0 109 0 0 0 19 0 5 0 501525671 289976320 24027 33554432000 4194304 4338100 140737150698064 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=283180 CPUtime=1.08 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 4096 0 0 0 108 0 0 0 19 0 5 0 501525671 289976320 24027 33554432000 4194304 4338100 140737150698064 18446744073709551615 4206902 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=1713/tid=1723] ppid=1708 vsize=283180 CPUtime=1.09 cores=0,2,4,6
/proc/1713/task/1723/stat : 1723 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 4437 0 0 0 109 0 0 0 25 0 5 0 501525671 289976320 24028 33554432000 4194304 4338100 140737150698064 18446744073709551615 4287096 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4.78
Current children cumulated vsize (KiB) 283180

[startup+3.10099 s]
/proc/loadavg: 3.33 6.01 7.01 10/186 1723
/proc/meminfo: memFree=30682920/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=365168 CPUtime=11.18 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) S 1708 1713 1676 0 -1 4202496 48288 0 0 0 1111 7 0 0 24 0 5 0 501525631 373932032 45673 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 91292 45673 176 36 0 89197 0
[pid=1713/tid=1720] ppid=1708 vsize=365168 CPUtime=2.7 cores=0,2,4,6
/proc/1713/task/1720/stat : 1720 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 10861 0 0 0 268 2 0 0 23 0 5 0 501525671 373932032 45673 33554432000 4194304 4338100 140737150698064 18446744073709551615 4256139 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=1713/tid=1721] ppid=1708 vsize=365168 CPUtime=2.69 cores=0,2,4,6
/proc/1713/task/1721/stat : 1721 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 11114 0 0 0 268 1 0 0 23 0 5 0 501525671 373932032 45673 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=365168 CPUtime=2.68 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 8698 0 0 0 267 1 0 0 23 0 5 0 501525671 373932032 45673 33554432000 4194304 4338100 140737150698064 18446744073709551615 4239484 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=1713/tid=1723] ppid=1708 vsize=365168 CPUtime=2.69 cores=0,2,4,6
/proc/1713/task/1723/stat : 1723 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 9319 0 0 0 268 1 0 0 25 0 5 0 501525671 373932032 45673 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 11.18
Current children cumulated vsize (KiB) 365168

[startup+6.3004 s]
/proc/loadavg: 3.70 6.05 7.01 10/186 1723
/proc/meminfo: memFree=30393256/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=531744 CPUtime=23.96 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) S 1708 1713 1676 0 -1 4202496 89384 0 0 0 2381 15 0 0 24 0 5 0 501525631 544505856 86769 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 132936 86769 176 36 0 130841 0
[pid=1713/tid=1720] ppid=1708 vsize=531744 CPUtime=5.88 cores=0,2,4,6
/proc/1713/task/1720/stat : 1720 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 19728 0 0 0 585 3 0 0 25 0 5 0 501525671 544505856 86769 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=1713/tid=1721] ppid=1708 vsize=531744 CPUtime=5.89 cores=0,2,4,6
/proc/1713/task/1721/stat : 1721 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 23426 0 0 0 586 3 0 0 25 0 5 0 501525671 544505856 86769 33554432000 4194304 4338100 140737150698064 18446744073709551615 4284173 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=531744 CPUtime=5.87 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 17803 0 0 0 584 3 0 0 25 0 5 0 501525671 544505856 86769 33554432000 4194304 4338100 140737150698064 18446744073709551615 4256993 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=1713/tid=1723] ppid=1708 vsize=531744 CPUtime=5.89 cores=0,2,4,6
/proc/1713/task/1723/stat : 1723 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 20131 0 0 0 586 3 0 0 25 0 5 0 501525671 544505856 86769 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 23.96
Current children cumulated vsize (KiB) 531744

[startup+12.7012 s]
/proc/loadavg: 4.04 6.08 7.02 10/186 1723
/proc/meminfo: memFree=29948440/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=776520 CPUtime=49.53 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) S 1708 1713 1676 0 -1 4202496 150655 0 0 0 4929 24 0 0 24 0 5 0 501525631 795156480 148040 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 194130 148040 176 36 0 192035 0
[pid=1713/tid=1720] ppid=1708 vsize=776520 CPUtime=12.25 cores=0,2,4,6
/proc/1713/task/1720/stat : 1720 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 32590 0 0 0 1219 6 0 0 25 0 5 0 501525671 795156480 148040 33554432000 4194304 4338100 140737150698064 18446744073709551615 4286315 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=1713/tid=1721] ppid=1708 vsize=776520 CPUtime=12.29 cores=0,2,4,6
/proc/1713/task/1721/stat : 1721 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 41716 0 0 0 1223 6 0 0 25 0 5 0 501525671 795156480 148040 33554432000 4194304 4338100 140737150698064 18446744073709551615 4284188 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=776520 CPUtime=12.27 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 34193 0 0 0 1222 5 0 0 25 0 5 0 501525671 795156480 148040 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=1713/tid=1723] ppid=1708 vsize=776520 CPUtime=12.29 cores=0,2,4,6
/proc/1713/task/1723/stat : 1723 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 33860 0 0 0 1224 5 0 0 25 0 5 0 501525671 795156480 148040 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 49.53
Current children cumulated vsize (KiB) 776520

[startup+25.5009 s]
/proc/loadavg: 4.92 6.17 7.03 10/186 1723
/proc/meminfo: memFree=29295000/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=1149676 CPUtime=100.65 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) S 1708 1713 1676 0 -1 4202496 244148 0 0 0 10024 41 0 0 24 0 5 0 501525631 1177268224 241533 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 287419 241533 176 36 0 285324 0
[pid=1713/tid=1720] ppid=1708 vsize=1149676 CPUtime=24.98 cores=0,2,4,6
/proc/1713/task/1720/stat : 1720 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 51607 0 0 0 2488 10 0 0 25 0 5 0 501525671 1177268224 241533 33554432000 4194304 4338100 140737150698064 18446744073709551615 4287678 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=1713/tid=1721] ppid=1708 vsize=1149676 CPUtime=25.09 cores=0,2,4,6
/proc/1713/task/1721/stat : 1721 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 76134 0 0 0 2498 11 0 0 25 0 5 0 501525671 1177268224 241534 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=1149676 CPUtime=25.07 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 56334 0 0 0 2498 9 0 0 25 0 5 0 501525671 1177268224 241534 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=1713/tid=1723] ppid=1708 vsize=1149676 CPUtime=25.09 cores=0,2,4,6
/proc/1713/task/1723/stat : 1723 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 51779 0 0 0 2501 8 0 0 25 0 5 0 501525671 1177268224 241535 33554432000 4194304 4338100 140737150698064 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 100.65
Current children cumulated vsize (KiB) 1149676

[startup+51.1011 s]
/proc/loadavg: 5.97 6.32 7.06 10/186 1724
/proc/meminfo: memFree=28397672/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=1699624 CPUtime=202.91 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) S 1708 1713 1676 0 -1 4202496 380349 0 0 0 20225 66 0 0 24 0 5 0 501525631 1740414976 377734 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 424906 377735 176 36 0 422811 0
[pid=1713/tid=1720] ppid=1708 vsize=1699624 CPUtime=50.44 cores=0,2,4,6
/proc/1713/task/1720/stat : 1720 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 73787 0 0 0 5030 14 0 0 25 0 5 0 501525671 1740414976 377735 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=1713/tid=1721] ppid=1708 vsize=1699624 CPUtime=50.69 cores=0,2,4,6
/proc/1713/task/1721/stat : 1721 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 118265 0 0 0 5051 18 0 0 25 0 5 0 501525671 1740414976 377736 33554432000 4194304 4338100 140737150698064 18446744073709551615 4263517 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=1699624 CPUtime=50.67 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 88342 0 0 0 5050 17 0 0 25 0 5 0 501525671 1740414976 377736 33554432000 4194304 4338100 140737150698064 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=1713/tid=1723] ppid=1708 vsize=1699624 CPUtime=50.69 cores=0,2,4,6
/proc/1713/task/1723/stat : 1723 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 91661 0 0 0 5054 15 0 0 25 0 5 0 501525671 1740414976 377736 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254291 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 202.91
Current children cumulated vsize (KiB) 1699624

[startup+102.307 s]
/proc/loadavg: 7.19 6.60 7.11 10/186 1725
/proc/meminfo: memFree=27020256/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=2576728 CPUtime=407.45 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) S 1708 1713 1676 0 -1 4202496 599673 0 0 0 40638 107 0 0 24 0 5 0 501525631 2638569472 597058 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 644182 597058 176 36 0 642087 0
[pid=1713/tid=1720] ppid=1708 vsize=2576728 CPUtime=101.36 cores=0,2,4,6
/proc/1713/task/1720/stat : 1720 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 110458 0 0 0 10116 20 0 0 25 0 5 0 501525671 2638569472 597058 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828049102 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=1713/tid=1721] ppid=1708 vsize=2576728 CPUtime=101.89 cores=0,2,4,6
/proc/1713/task/1721/stat : 1721 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 181677 0 0 0 10161 28 0 0 25 0 5 0 501525671 2638569472 597058 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828049111 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=2576728 CPUtime=101.87 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 122952 0 0 0 10162 25 0 0 25 0 5 0 501525671 2638569472 597058 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=1713/tid=1723] ppid=1708 vsize=2576728 CPUtime=101.89 cores=0,2,4,6
/proc/1713/task/1723/stat : 1723 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 176290 0 0 0 10158 31 0 0 25 0 5 0 501525671 2638569472 597058 33554432000 4194304 4338100 140737150698064 18446744073709551615 4287681 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 407.45
Current children cumulated vsize (KiB) 2576728

[startup+162.301 s]
/proc/loadavg: 7.77 6.88 7.18 10/186 1727
/proc/meminfo: memFree=25820512/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=3312996 CPUtime=647.09 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) S 1708 1713 1676 0 -1 4202496 785134 0 0 0 64566 143 0 0 24 0 5 0 501525631 3392507904 782519 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 828249 782519 176 36 0 826154 0
[pid=1713/tid=1720] ppid=1708 vsize=3312996 CPUtime=161.04 cores=0,2,4,6
/proc/1713/task/1720/stat : 1720 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 145838 0 0 0 16076 28 0 0 25 0 5 0 501525671 3392507904 782519 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=1713/tid=1721] ppid=1708 vsize=3312996 CPUtime=161.88 cores=0,2,4,6
/proc/1713/task/1721/stat : 1721 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 240307 0 0 0 16150 38 0 0 25 0 5 0 501525671 3392507904 782519 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=3312996 CPUtime=161.87 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 157197 0 0 0 16155 32 0 0 25 0 5 0 501525671 3392507904 782519 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=1713/tid=1723] ppid=1708 vsize=3312996 CPUtime=161.88 cores=0,2,4,6
/proc/1713/task/1723/stat : 1723 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 233496 0 0 0 16145 43 0 0 25 0 5 0 501525671 3392507904 782519 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 647.09
Current children cumulated vsize (KiB) 3312996


################
# More data... #
################

/proc/1713/task/1723/stat : 1723 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 587417 0 0 0 75787 119 0 0 25 0 5 0 501525671 7729037312 1834678 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3043.8
Current children cumulated vsize (KiB) 7547888

[startup+822.301 s]
/proc/loadavg: 8.13 7.96 7.62 10/186 1745
/proc/meminfo: memFree=19064676/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=7770504 CPUtime=3283.45 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) S 1708 1713 1676 0 -1 4202496 1893692 0 0 0 327964 381 0 0 24 0 5 0 501525631 7956996096 1891077 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 1942626 1891077 176 36 0 1940531 0
[pid=1713/tid=1720] ppid=1708 vsize=7770504 CPUtime=820.78 cores=0,2,4,6
/proc/1713/task/1720/stat : 1720 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 303937 0 0 0 82014 64 0 0 25 0 5 0 501525671 7956996096 1891077 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=1713/tid=1721] ppid=1708 vsize=7770504 CPUtime=821.88 cores=0,2,4,6
/proc/1713/task/1721/stat : 1721 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 577676 0 0 0 82082 106 0 0 25 0 5 0 501525671 7956996096 1891077 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=7770504 CPUtime=821.31 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 409301 0 0 0 82043 88 0 0 25 0 5 0 501525671 7956996096 1891077 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254275 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=1713/tid=1723] ppid=1708 vsize=7770504 CPUtime=819.06 cores=0,2,4,6
/proc/1713/task/1723/stat : 1723 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 594482 0 0 0 81785 121 0 0 25 0 5 0 501525671 7956996096 1891077 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3283.45
Current children cumulated vsize (KiB) 7770504

[startup+882.301 s]
/proc/loadavg: 8.12 7.99 7.65 10/186 1746
/proc/meminfo: memFree=18665624/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=7984492 CPUtime=3523.07 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) S 1708 1713 1676 0 -1 4202496 1943445 0 0 0 351916 391 0 0 24 0 5 0 501525631 8176119808 1940830 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 1996123 1940830 176 36 0 1994028 0
[pid=1713/tid=1720] ppid=1708 vsize=7984492 CPUtime=880.78 cores=0,2,4,6
/proc/1713/task/1720/stat : 1720 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 313483 0 0 0 88012 66 0 0 25 0 5 0 501525671 8176119808 1940830 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=1713/tid=1721] ppid=1708 vsize=7984492 CPUtime=881.85 cores=0,2,4,6
/proc/1713/task/1721/stat : 1721 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 593312 0 0 0 88076 109 0 0 25 0 5 0 501525671 8176119808 1940830 33554432000 4194304 4338100 140737150698064 18446744073709551615 4239472 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=7984492 CPUtime=880.97 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 423373 0 0 0 88006 91 0 0 25 0 5 0 501525671 8176119808 1940830 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254433 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=1713/tid=1723] ppid=1708 vsize=7984492 CPUtime=879.05 cores=0,2,4,6
/proc/1713/task/1723/stat : 1723 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 604981 0 0 0 87782 123 0 0 25 0 5 0 501525671 8176119808 1940830 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3523.07
Current children cumulated vsize (KiB) 7984492

[startup+942.301 s]
/proc/loadavg: 8.04 7.99 7.67 10/186 1748
/proc/meminfo: memFree=18446724/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=8069476 CPUtime=3762.75 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) S 1708 1713 1676 0 -1 4202496 1965522 0 0 0 375879 396 0 0 24 0 5 0 501525631 8263143424 1962907 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 2017369 1962907 176 36 0 2015274 0
[pid=1713/tid=1720] ppid=1708 vsize=8069476 CPUtime=940.77 cores=0,2,4,6
/proc/1713/task/1720/stat : 1720 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 313510 0 0 0 94011 66 0 0 25 0 5 0 501525671 8263143424 1962907 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=1713/tid=1721] ppid=1708 vsize=8069476 CPUtime=941.85 cores=0,2,4,6
/proc/1713/task/1721/stat : 1721 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 612473 0 0 0 94073 112 0 0 25 0 5 0 501525671 8263143424 1962907 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=8069476 CPUtime=940.65 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 423431 0 0 0 93974 91 0 0 25 0 5 0 501525671 8263143424 1962907 33554432000 4194304 4338100 140737150698064 18446744073709551615 4285383 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=1713/tid=1723] ppid=1708 vsize=8069476 CPUtime=939.05 cores=0,2,4,6
/proc/1713/task/1723/stat : 1723 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 607812 0 0 0 93781 124 0 0 25 0 5 0 501525671 8263143424 1962907 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3762.75
Current children cumulated vsize (KiB) 8069476

[startup+1002.3 s]
/proc/loadavg: 8.01 7.99 7.68 10/186 1749
/proc/meminfo: memFree=18316108/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=8229696 CPUtime=4002.42 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) S 1708 1713 1676 0 -1 4202496 1996949 0 0 0 399838 404 0 0 24 0 5 0 501525631 8427208704 1994334 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 2057424 1994334 176 36 0 2055329 0
[pid=1713/tid=1720] ppid=1708 vsize=8229696 CPUtime=1000.77 cores=0,2,4,6
/proc/1713/task/1720/stat : 1720 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 316068 0 0 0 100010 67 0 0 25 0 5 0 501525671 8427208704 1994334 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=1713/tid=1721] ppid=1708 vsize=8229696 CPUtime=1001.85 cores=0,2,4,6
/proc/1713/task/1721/stat : 1721 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 628609 0 0 0 100070 115 0 0 25 0 5 0 501525671 8427208704 1994334 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254357 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=8229696 CPUtime=1000.32 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 423483 0 0 0 99941 91 0 0 25 0 5 0 501525671 8427208704 1994334 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=1713/tid=1723] ppid=1708 vsize=8229696 CPUtime=999.05 cores=0,2,4,6
/proc/1713/task/1723/stat : 1723 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 620493 0 0 0 99777 128 0 0 25 0 5 0 501525671 8427208704 1994334 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4002.42
Current children cumulated vsize (KiB) 8229696

[startup+1062.3 s]
/proc/loadavg: 8.00 7.99 7.70 10/186 1751
/proc/meminfo: memFree=18247480/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=8292652 CPUtime=4242.1 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) S 1708 1713 1676 0 -1 4202496 2014212 0 0 0 423802 408 0 0 24 0 5 0 501525631 8491675648 2011597 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 2073163 2011597 176 36 0 2071068 0
[pid=1713/tid=1720] ppid=1708 vsize=8292652 CPUtime=1060.77 cores=0,2,4,6
/proc/1713/task/1720/stat : 1720 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 317240 0 0 0 106009 68 0 0 25 0 5 0 501525671 8491675648 2011597 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254275 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=1713/tid=1721] ppid=1708 vsize=8292652 CPUtime=1061.85 cores=0,2,4,6
/proc/1713/task/1721/stat : 1721 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 637377 0 0 0 106069 116 0 0 25 0 5 0 501525671 8491675648 2011597 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=8292652 CPUtime=1060 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 423533 0 0 0 105908 92 0 0 25 0 5 0 501525671 8491675648 2011597 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=1713/tid=1723] ppid=1708 vsize=8292652 CPUtime=1059.06 cores=0,2,4,6
/proc/1713/task/1723/stat : 1723 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 627766 0 0 0 105776 130 0 0 25 0 5 0 501525671 8491675648 2011597 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254418 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4242.1
Current children cumulated vsize (KiB) 8292652

[startup+1122.3 s]
/proc/loadavg: 8.11 8.02 7.73 10/186 1753
/proc/meminfo: memFree=18052148/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=8462092 CPUtime=4481.76 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) S 1708 1713 1676 0 -1 4202496 2057411 0 0 0 447759 417 0 0 24 0 5 0 501525631 8665182208 2054796 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 2115523 2054796 176 36 0 2113428 0
[pid=1713/tid=1720] ppid=1708 vsize=8462092 CPUtime=1120.76 cores=0,2,4,6
/proc/1713/task/1720/stat : 1720 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 317283 0 0 0 112008 68 0 0 25 0 5 0 501525671 8665182208 2054796 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=1713/tid=1721] ppid=1708 vsize=8462092 CPUtime=1121.86 cores=0,2,4,6
/proc/1713/task/1721/stat : 1721 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 670504 0 0 0 112063 123 0 0 25 0 5 0 501525671 8665182208 2054796 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=8462092 CPUtime=1119.67 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 423571 0 0 0 111875 92 0 0 25 0 5 0 501525671 8665182208 2054796 33554432000 4194304 4338100 140737150698064 18446744073709551615 4256139 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=1713/tid=1723] ppid=1708 vsize=8462092 CPUtime=1119.05 cores=0,2,4,6
/proc/1713/task/1723/stat : 1723 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 637757 0 0 0 111773 132 0 0 25 0 5 0 501525671 8665182208 2054796 33554432000 4194304 4338100 140737150698064 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4481.76
Current children cumulated vsize (KiB) 8462092

[startup+1182.3 s]
/proc/loadavg: 8.11 8.03 7.75 10/186 1754
/proc/meminfo: memFree=17680612/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=8722244 CPUtime=4721.45 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) S 1708 1713 1676 0 -1 4202496 2122433 0 0 0 471711 434 0 0 24 0 5 0 501525631 8931577856 2119818 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 2180561 2119818 176 36 0 2178466 0
[pid=1713/tid=1720] ppid=1708 vsize=8722244 CPUtime=1180.76 cores=0,2,4,6
/proc/1713/task/1720/stat : 1720 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 317310 0 0 0 118008 68 0 0 25 0 5 0 501525671 8931577856 2119818 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=1713/tid=1721] ppid=1708 vsize=8722244 CPUtime=1181.86 cores=0,2,4,6
/proc/1713/task/1721/stat : 1721 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 711184 0 0 0 118054 132 0 0 25 0 5 0 501525671 8931577856 2119818 33554432000 4194304 4338100 140737150698064 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=8722244 CPUtime=1179.34 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 430040 0 0 0 117841 93 0 0 25 0 5 0 501525671 8931577856 2119818 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254239 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=1713/tid=1723] ppid=1708 vsize=8722244 CPUtime=1179.06 cores=0,2,4,6
/proc/1713/task/1723/stat : 1723 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 655603 0 0 0 117768 138 0 0 25 0 5 0 501525671 8931577856 2119818 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4721.45
Current children cumulated vsize (KiB) 8722244



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202 s]
/proc/loadavg: 8.08 8.03 7.75 6/184 1756
/proc/meminfo: memFree=20483992/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=8825732 CPUtime=4800.14 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) S 1708 1713 1676 0 -1 4202496 2148236 0 0 0 479576 438 0 0 24 0 5 0 501525631 9037549568 2145621 33554432000 4194304 4338100 140737150698064 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 2206433 2145621 176 36 0 2204338 0
[pid=1713/tid=1720] ppid=1708 vsize=8825732 CPUtime=1200.46 cores=0,2,4,6
/proc/1713/task/1720/stat : 1720 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 317317 0 0 0 119978 68 0 0 25 0 5 0 501525671 9037549568 2145621 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=1713/tid=1721] ppid=1708 vsize=8825732 CPUtime=1201.56 cores=0,2,4,6
/proc/1713/task/1721/stat : 1721 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 724943 0 0 0 120022 134 0 0 25 0 5 0 501525671 9037549568 2145621 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=8825732 CPUtime=1198.93 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 435453 0 0 0 119799 94 0 0 25 0 5 0 501525671 9037549568 2145621 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=1713/tid=1723] ppid=1708 vsize=8825732 CPUtime=1198.76 cores=0,2,4,6
/proc/1713/task/1723/stat : 1723 (ttsth-5-4) R 1708 1713 1676 0 -1 4202560 662227 0 0 0 119737 139 0 0 25 0 5 0 501525671 9037549568 2145621 33554432000 4194304 4338100 140737150698064 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.14
Current children cumulated vsize (KiB) 8825732

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202.2 s]
/proc/loadavg: 8.08 8.03 7.75 2/178 1757
/proc/meminfo: memFree=26747776/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=0 CPUtime=4800.36 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) Z 1708 1713 1676 0 -1 4203548 2148238 0 0 0 479579 457 0 0 18 0 2 0 501525631 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 0 0 0 0 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=0 CPUtime=1199.14 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4203588 435455 0 0 0 119800 114 0 0 25 0 2 0 501525671 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.36
Current children cumulated vsize (KiB) 0

[startup+1202.31 s]
/proc/loadavg: 8.08 8.03 7.75 2/178 1757
/proc/meminfo: memFree=26747776/32951124 swapFree=67105204/67111528
[pid=1713] ppid=1708 vsize=0 CPUtime=4800.47 cores=0,2,4,6
/proc/1713/stat : 1713 (ttsth-5-4) Z 1708 1713 1676 0 -1 4203548 2148238 0 0 0 479579 468 0 0 18 0 2 0 501525631 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/1713/statm: 0 0 0 0 0 0 0
[pid=1713/tid=1722] ppid=1708 vsize=0 CPUtime=1199.24 cores=0,2,4,6
/proc/1713/task/1722/stat : 1722 (ttsth-5-4) R 1708 1713 1676 0 -1 4203588 435455 0 0 0 119800 124 0 0 25 0 2 0 501525671 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.47
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.37
CPU time (s): 4800.54
CPU user time (s): 4795.8
CPU system time (s): 4.74328
CPU usage (%): 399.257
Max. virtual memory (cumulated for all children) (KiB): 8825732

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4795.8
system time used= 4.74328
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2148238
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 883
involuntary context switches= 41582

runsolver used 1.76673 second user time and 4.78827 second system time

The end

Launcher Data 

Begin job on node147 at 2011-04-29 17:31:37
IDJOB=3262090
IDBENCH=83272
IDSOLVER=1654
FILE ID=node147/3262090-1304091097
RUNJOBID= node147-1304091097-1694
PBS_JOBID= 13170578
Free space on /tmp= 72660 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/kullmann/VanDerWaerden/VanderWaerden_pd_3k/VanDerWaerden_pd_2-3-21_404.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3262090-1304091097/watcher-3262090-1304091097 -o /tmp/evaluation-result-3262090-1304091097/solver-3262090-1304091097 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262090-1304091097.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 213e4c0c58f043e67e86cf993698317a
RANDOM SEED=378769244

node147.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30926328 kB
Buffers:        273196 kB
Cached:        1365216 kB
SwapCached:       2968 kB
Active:         125276 kB
Inactive:      1574060 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30926328 kB
SwapTotal:    67111528 kB
SwapFree:     67105204 kB
Dirty:             880 kB
Writeback:           0 kB
AnonPages:       60100 kB
Mapped:          14240 kB
Slab:           260508 kB
PageTables:       5044 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   184776 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72660 MiB
End job on node147 at 2011-04-29 17:51:41