Trace number 3251484

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.47 1204.55

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m42-q663-n40-p15-s72490337.cnf
MD5SUMab8e40d31c79e7a71114e3126a46874f
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark1.38479
Satisfiable
(Un)Satisfiability was proved
Number of variables1649
Number of clauses256038
Sum of the clauses size513645
Maximum clause length42
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2255998
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 540

Solver Data 

0.09/0.17	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3251484-1304107892.cnf 
0.09/0.17	c Using SATTIMEOUT of 4800
0.09/0.17	c Using SATRAM of 15500
0.09/0.17	c Size of subset node = 32
0.09/0.17	c Variables = 1649
0.09/0.17	c Clauses = 256038
0.09/0.17	c Literals = 513645
0.09/0.17	c Reduced size:
0.09/0.17	c Variables = 1649
0.09/0.17	c Clauses = 256038
0.09/0.17	c Literals = 513645
0.09/0.17	c a2
0.29/0.31	c a1
0.69/0.71	c Independent sub-problems = 1
0.69/0.71	c Sub-problem with 1649 variables and 256038 clauses
0.69/0.71	c using permutation method 4
0.69/0.71	c count = 1
0.69/0.71	c Ordering finished 0.670 seconds
0.69/0.71	c mapped 0.700 seconds
0.69/0.71	c permuted 0.700 seconds
0.69/0.71	c about to sort
0.69/0.73	c sorted
1.39/1.47	c clauses now 256038
1.39/1.47	c init finished 0.790 seconds
1.39/1.47	c tree built 1.460 seconds
1.39/1.47	c 
1.39/1.47	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
1.39/1.47	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
1.39/1.47	c ................................ | ..........                    |                                  1.5         0.0             
1.39/1.47	c ................................ | ..........                  1 |                                  1.5         0.7             
1.39/1.47	c ................................ | ..........                  2 |                                  1.5         1.4             
1.39/1.47	c ................................ | ..........                  3 |                                  1.5         2.1             
1.39/1.47	c ................................ | ..........                  4 |                                  1.5         2.7             
1.39/1.47	c ................................ | ..........                  5 |                                  1.5         3.4             
1.39/1.47	c ................................ | ..........               1000 |              52                  1.5       680.3         1223
95.84/25.14	c ................................ | ..........            5754000 | 4955         607702             96.0     59937.5     20135196
191.69/49.18	c ................................ | ..........           11234000 | 8751         1196510           192.0     58507.4     35718889
287.93/73.21	c ................................ | ..........           16349000 | 12530        1747761           288.0     56767.4     43328981
383.76/97.26	c ................................ | ..........           21780000 | 14465        2308848           384.0     56717.3     53510446
479.61/121.29	c ................................ | *.........           26914000 | 17953        2854188           480.0     56070.8     58847468
575.84/145.34	c ................................ | *.........           32185000 | 20978        3401487           576.0     55875.8     70093403
671.70/169.38	c ................................ | *.........           37438000 | 24123        3949910           672.0     55710.5     77613963
767.91/193.42	c ................................ | *.........           42685000 | 26456        4495017           768.0     55578.7     86622346
863.71/217.47	c ................................ | *.........           47758000 | 27649        5024567           864.0     55275.5     95579159
959.90/241.52	c ................................ | **........           52794000 | 30295        5551583           960.0     54993.8    100004907
1055.68/265.58	c ................................ | **........           57713000 | 33405        6075090          1056.0     54651.9    107832934
1152.25/289.71	c ................................ | **........           62690000 | 35030        6592403          1152.3     54404.2    119157411
1247.63/313.69	c ................................ | **........           67625000 | 36786        7098349          1248.0     54186.7    126580630
1343.85/337.74	c ................................ | **........           72546000 | 37973        7599968          1344.0     53977.7    136141144
1440.04/361.82	c ................................ | ***.......           77370000 | 40807        8093886          1440.1     53725.1    141578688
1535.74/385.86	c ................................ | ***.......           82045000 | 43074        8575616          1536.0     53414.4    142548526
1631.94/409.91	c ................................ | ***.......           86819000 | 45259        9056225          1632.0     53197.9    150423298
1727.75/433.96	c ................................ | ***.......           91414000 | 47058        9531450          1728.0     52901.3    155108297
1823.87/458.04	c ................................ | ***.......           95899000 | 49106        10008181         1824.0     52575.9    160669314
1919.96/482.11	c ................................ | ****......          100253000 | 50622        10476647         1920.0     52215.1    163758923
2015.76/506.16	c ................................ | ****......          104658000 | 52940        10953571         2016.0     51913.4    169899493
2111.95/530.22	c ................................ | ****......          109077000 | 54468        11433566         2112.0     51645.8    178608218
2207.72/554.27	c ................................ | ****......          113556000 | 55602        11907299         2208.0     51429.1    184555603
2303.91/578.32	c ................................ | ****......          117984000 | 56372        12376929         2304.0     51208.1    189236707
2399.71/602.37	c ................................ | *****.....          122476000 | 57876        12848489         2400.0     51031.5    197425422
2495.89/626.42	c ................................ | *****.....          126830000 | 59275        13321567         2496.0     50813.3    201480199
2591.67/650.48	c ................................ | *****.....          131323000 | 61149        13808277         2592.0     50664.5    209779573
2687.85/674.54	c ................................ | *****.....          135881000 | 63953        14318065         2688.0     50551.0    217877206
2783.98/698.60	c ................................ | *****.....          140416000 | 65097        14807726         2784.0     50436.8    226760546
2879.79/722.66	c ................................ | ******....          144626000 | 66259        15279499         2880.0     50217.4    226760546
2975.95/746.71	c ................................ | ******....          148742000 | 69357        15735829         2976.0     49980.5    229185751
3071.70/770.77	c ................................ | ******....          153085000 | 71644        16219741         3072.0     49832.4    237013751
3167.82/794.84	c ................................ | ******....          157551000 | 75380        16701098         3168.0     49731.9    246036606
3263.93/818.92	c ................................ | ******....          161627000 | 78323        17157259         3264.0     49517.9    248610417
3359.65/842.99	c ................................ | *******...          165845000 | 80308        17619873         3360.0     49358.5    254565543
3455.72/867.07	c ................................ | *******...          170090000 | 83550        18082722         3456.0     49215.9    262623279
3551.80/891.14	c ................................ | *******...          174189000 | 85533        18521633         3552.0     49039.7    264069457
3648.01/915.20	c ................................ | *******...          178185000 | 86936        18964286         3648.0     48844.4    265573077
3743.74/939.26	c ................................ | *******...          182527000 | 88779        19418408         3744.0     48751.7    275731288
3840.66/963.59	c ................................ | ********..          186700000 | 91260        19868964         3841.0     48607.0    284410381
3935.97/987.41	c ................................ | ********..          190796000 | 92870        20308633         3936.0     48474.5    285367865
4031.75/1011.46	c ................................ | ********..          194911000 | 94202        20743210         4032.0     48340.9    292906007
4127.86/1035.53	c ................................ | ********..          198906000 | 95816        21168262         4128.0     48184.6    295628010
4223.63/1059.59	c ................................ | ********..          202872000 | 98097        21597657         4224.0     48028.4    297277381
4319.75/1083.66	c ................................ | *********.          207142000 | 99897        22046209         4320.0     47949.4    306457477
4415.88/1107.74	c ................................ | *********.          211298000 | 102521       22486744         4416.0     47848.2    312998061
4511.99/1131.81	c ................................ | *********.          215632000 | 105365       22946805         4512.0     47790.7    322224525
4607.72/1155.87	c ................................ | *********.          220195000 | 110492       23398055         4608.0     47785.4    327556166
4704.18/1180.01	c ................................ | *********.          224680000 | 114413       23838263         4704.2     47761.4    330077276
4799.69/1204.07	c CPU limit (4800) reached
4799.69/1204.07	c Time = 4800.00 seconds
4799.69/1204.07	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3251484-1304107892/watcher-3251484-1304107892 -o /tmp/evaluation-result-3251484-1304107892/solver-3251484-1304107892 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3251484-1304107892.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.07 7.28 7.71 2/177 30663
/proc/meminfo: memFree=17421452/32950940 swapFree=67111528/67111528
[pid=30663] ppid=30661 vsize=9764 CPUtime=0 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) R 30661 30663 30624 0 -1 4202496 490 0 0 0 0 0 0 0 25 0 1 0 601020005 9998336 432 33554432000 4194304 4338100 140734190810848 18446744073709551615 215988592384 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/30663/statm: 2441 432 147 36 0 346 0

[startup+0.050447 s]
/proc/loadavg: 5.07 7.28 7.71 2/177 30663
/proc/meminfo: memFree=17421452/32950940 swapFree=67111528/67111528
[pid=30663] ppid=30661 vsize=18672 CPUtime=0.04 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) R 30661 30663 30624 0 -1 4202496 2721 0 0 0 4 0 0 0 25 0 1 0 601020005 19120128 2663 33554432000 4194304 4338100 140734190810848 18446744073709551615 215988253381 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/30663/statm: 4668 2664 148 36 0 2573 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 18672

[startup+0.100437 s]
/proc/loadavg: 5.07 7.28 7.71 2/177 30663
/proc/meminfo: memFree=17421452/32950940 swapFree=67111528/67111528
[pid=30663] ppid=30661 vsize=28944 CPUtime=0.09 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) R 30661 30663 30624 0 -1 4202496 5225 0 0 0 9 0 0 0 25 0 1 0 601020005 29638656 5167 33554432000 4194304 4338100 140734190810848 18446744073709551615 215988253381 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/30663/statm: 7236 5169 148 36 0 5141 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 28944

[startup+0.300404 s]
/proc/loadavg: 5.07 7.28 7.71 2/177 30663
/proc/meminfo: memFree=17421452/32950940 swapFree=67111528/67111528
[pid=30663] ppid=30661 vsize=59676 CPUtime=0.29 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) R 30661 30663 30624 0 -1 4202496 12346 0 0 0 28 1 0 0 25 0 1 0 601020005 61108224 12287 33554432000 4194304 4338100 140734190810848 18446744073709551615 215987987650 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/30663/statm: 14919 12287 153 36 0 12824 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 59676

[startup+0.700337 s]
/proc/loadavg: 5.07 7.28 7.71 2/177 30663
/proc/meminfo: memFree=17421452/32950940 swapFree=67111528/67111528
[pid=30663] ppid=30661 vsize=66108 CPUtime=0.69 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) R 30661 30663 30624 0 -1 4202496 17251 0 0 0 68 1 0 0 25 0 1 0 601020005 67694592 14561 33554432000 4194304 4338100 140734190810848 18446744073709551615 215987988343 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/30663/statm: 16527 14561 161 36 0 14432 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 66108

[startup+1.5002 s]
/proc/loadavg: 5.07 7.28 7.71 3/178 30664
/proc/meminfo: memFree=17240536/32950940 swapFree=67111528/67111528
[pid=30663] ppid=30661 vsize=330244 CPUtime=1.56 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) S 30661 30663 30624 0 -1 4202496 38627 0 0 0 152 4 0 0 25 0 5 0 601020005 338169856 34935 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 82561 34935 176 36 0 80466 0
Current children cumulated CPU time (s) 1.56
Current children cumulated vsize (KiB) 330244

[startup+3.10092 s]
/proc/loadavg: 5.30 7.29 7.71 10/186 30672
/proc/meminfo: memFree=17106800/32950940 swapFree=67111528/67111528
[pid=30663] ppid=30661 vsize=388180 CPUtime=7.96 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) S 30661 30663 30624 0 -1 4202496 53557 0 0 0 775 21 0 0 25 0 5 0 601020005 397496320 49865 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 97045 49866 176 36 0 94950 0
[pid=30663/tid=30669] ppid=30661 vsize=388180 CPUtime=1.62 cores=1,3,5,7
/proc/30663/task/30669/stat : 30669 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 4978 0 0 0 161 1 0 0 25 0 5 0 601020153 397496320 49866 33554432000 4194304 4338100 140734190810848 18446744073709551615 4256015 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=388180 CPUtime=1.61 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 2804 0 0 0 155 6 0 0 25 0 5 0 601020153 397496320 49867 33554432000 4194304 4338100 140734190810848 18446744073709551615 4239487 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30663/tid=30671] ppid=30661 vsize=388180 CPUtime=1.62 cores=1,3,5,7
/proc/30663/task/30671/stat : 30671 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 5199 0 0 0 161 1 0 0 25 0 5 0 601020153 397496320 49867 33554432000 4194304 4338100 140734190810848 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30663/tid=30672] ppid=30661 vsize=388180 CPUtime=1.61 cores=1,3,5,7
/proc/30663/task/30672/stat : 30672 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 2647 0 0 0 154 7 0 0 25 0 5 0 601020153 397496320 49867 33554432000 4194304 4338100 140734190810848 18446744073709551615 4239487 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 7.96
Current children cumulated vsize (KiB) 388180

[startup+6.30036 s]
/proc/loadavg: 5.30 7.29 7.71 10/186 30672
/proc/meminfo: memFree=16861036/32950940 swapFree=67111528/67111528
[pid=30663] ppid=30661 vsize=501876 CPUtime=20.74 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) S 30661 30663 30624 0 -1 4202496 82458 0 0 0 2021 53 0 0 25 0 5 0 601020005 513921024 78766 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 125469 78766 176 36 0 123374 0
[pid=30663/tid=30669] ppid=30661 vsize=501876 CPUtime=4.82 cores=1,3,5,7
/proc/30663/task/30669/stat : 30669 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 13736 0 0 0 479 3 0 0 25 0 5 0 601020153 513921024 78766 33554432000 4194304 4338100 140734190810848 18446744073709551615 4265562 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=501876 CPUtime=4.8 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 8505 0 0 0 459 21 0 0 25 0 5 0 601020153 513921024 78766 33554432000 4194304 4338100 140734190810848 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30663/tid=30671] ppid=30661 vsize=501876 CPUtime=4.81 cores=1,3,5,7
/proc/30663/task/30671/stat : 30671 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 14306 0 0 0 479 2 0 0 25 0 5 0 601020153 513921024 78766 33554432000 4194304 4338100 140734190810848 18446744073709551615 4240496 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30663/tid=30672] ppid=30661 vsize=501876 CPUtime=4.8 cores=1,3,5,7
/proc/30663/task/30672/stat : 30672 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 7981 0 0 0 459 21 0 0 25 0 5 0 601020153 513921024 78766 33554432000 4194304 4338100 140734190810848 18446744073709551615 4239484 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 20.74
Current children cumulated vsize (KiB) 501876

[startup+12.7002 s]
/proc/loadavg: 5.72 7.31 7.71 9/186 30672
/proc/meminfo: memFree=16468316/32950940 swapFree=67111528/67111528
[pid=30663] ppid=30661 vsize=681600 CPUtime=46.3 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) S 30661 30663 30624 0 -1 4202496 126606 0 0 0 4508 122 0 0 25 0 5 0 601020005 697958400 122914 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 170400 122914 176 36 0 168305 0
[pid=30663/tid=30669] ppid=30661 vsize=681600 CPUtime=11.21 cores=1,3,5,7
/proc/30663/task/30669/stat : 30669 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 30108 0 0 0 1113 8 0 0 25 0 5 0 601020153 697958400 122914 33554432000 4194304 4338100 140734190810848 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=681600 CPUtime=11.2 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 12775 0 0 0 1067 53 0 0 25 0 5 0 601020153 697958400 122914 33554432000 4194304 4338100 140734190810848 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30663/tid=30671] ppid=30661 vsize=681600 CPUtime=11.21 cores=1,3,5,7
/proc/30663/task/30671/stat : 30671 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 30156 0 0 0 1117 4 0 0 25 0 5 0 601020153 697958400 122914 33554432000 4194304 4338100 140734190810848 18446744073709551615 4285854 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30663/tid=30672] ppid=30661 vsize=681600 CPUtime=11.17 cores=1,3,5,7
/proc/30663/task/30672/stat : 30672 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 15637 0 0 0 1066 51 0 0 25 0 5 0 601020153 697958400 122914 33554432000 4194304 4338100 140734190810848 18446744073709551615 4239444 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 46.3
Current children cumulated vsize (KiB) 681600

[startup+25.501 s]
/proc/loadavg: 6.15 7.35 7.72 10/186 30672
/proc/meminfo: memFree=15720192/32950940 swapFree=67111528/67111528
[pid=30663] ppid=30661 vsize=995104 CPUtime=97.44 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) S 30661 30663 30624 0 -1 4202496 205762 0 0 0 9484 260 0 0 25 0 5 0 601020005 1018986496 200782 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 248776 200782 176 36 0 246681 0
[pid=30663/tid=30669] ppid=30661 vsize=995104 CPUtime=24.01 cores=1,3,5,7
/proc/30663/task/30669/stat : 30669 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 63243 0 0 0 2384 17 0 0 25 0 5 0 601020153 1018986496 200782 33554432000 4194304 4338100 140734190810848 18446744073709551615 215987987924 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=995104 CPUtime=24 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 22474 0 0 0 2283 117 0 0 25 0 5 0 601020153 1018986496 200783 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000418485 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30663/tid=30671] ppid=30661 vsize=995104 CPUtime=24.01 cores=1,3,5,7
/proc/30663/task/30671/stat : 30671 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 58560 0 0 0 2392 9 0 0 25 0 5 0 601020153 1018986496 200783 33554432000 4194304 4338100 140734190810848 18446744073709551615 4288422 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30663/tid=30672] ppid=30661 vsize=995104 CPUtime=23.89 cores=1,3,5,7
/proc/30663/task/30672/stat : 30672 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 23555 0 0 0 2279 110 0 0 25 0 5 0 601020153 1018986496 200783 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000418485 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 97.44
Current children cumulated vsize (KiB) 995104

[startup+51.1005 s]
/proc/loadavg: 6.78 7.40 7.73 9/186 30673
/proc/meminfo: memFree=14756596/32950940 swapFree=67111528/67111528
[pid=30663] ppid=30661 vsize=1533980 CPUtime=199.68 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) S 30661 30663 30624 0 -1 4202496 340661 0 0 0 19425 543 0 0 25 0 5 0 601020005 1570795520 335681 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 383495 335681 176 36 0 381400 0
[pid=30663/tid=30669] ppid=30661 vsize=1533980 CPUtime=49.62 cores=1,3,5,7
/proc/30663/task/30669/stat : 30669 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 126297 0 0 0 4926 36 0 0 25 0 5 0 601020153 1570795520 335681 33554432000 4194304 4338100 140734190810848 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=1533980 CPUtime=49.46 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 31752 0 0 0 4702 244 0 0 25 0 5 0 601020153 1570795520 335682 33554432000 4194304 4338100 140734190810848 18446744073709551615 4283592 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30663/tid=30671] ppid=30661 vsize=1533980 CPUtime=49.62 cores=1,3,5,7
/proc/30663/task/30671/stat : 30671 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 103664 0 0 0 4945 17 0 0 25 0 5 0 601020153 1570795520 335682 33554432000 4194304 4338100 140734190810848 18446744073709551615 4257956 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30663/tid=30672] ppid=30661 vsize=1533980 CPUtime=49.48 cores=1,3,5,7
/proc/30663/task/30672/stat : 30672 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 41019 0 0 0 4708 240 0 0 25 0 5 0 601020153 1570795520 335682 33554432000 4194304 4338100 140734190810848 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 199.68
Current children cumulated vsize (KiB) 1533980

[startup+102.307 s]
/proc/loadavg: 7.52 7.50 7.74 9/186 30674
/proc/meminfo: memFree=13022676/32950940 swapFree=67111528/67111528
[pid=30663] ppid=30661 vsize=2156188 CPUtime=404.16 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) S 30661 30663 30624 0 -1 4202496 492068 0 0 0 39308 1108 0 0 25 0 5 0 601020005 2207936512 487088 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 539047 487088 176 36 0 536952 0
[pid=30663/tid=30669] ppid=30661 vsize=2156188 CPUtime=100.8 cores=1,3,5,7
/proc/30663/task/30669/stat : 30669 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 200422 0 0 0 10020 60 0 0 25 0 5 0 601020153 2207936512 487088 33554432000 4194304 4338100 140734190810848 18446744073709551615 4229558 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=2156188 CPUtime=100.39 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 39227 0 0 0 9523 516 0 0 25 0 5 0 601020153 2207936512 487088 33554432000 4194304 4338100 140734190810848 18446744073709551615 215987988206 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30663/tid=30671] ppid=30661 vsize=2156188 CPUtime=100.81 cores=1,3,5,7
/proc/30663/task/30671/stat : 30671 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 167323 0 0 0 10054 27 0 0 25 0 5 0 601020153 2207936512 487088 33554432000 4194304 4338100 140734190810848 18446744073709551615 215987987699 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30663/tid=30672] ppid=30661 vsize=2156188 CPUtime=100.66 cores=1,3,5,7
/proc/30663/task/30672/stat : 30672 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 47166 0 0 0 9567 499 0 0 25 0 5 0 601020153 2207936512 487088 33554432000 4194304 4338100 140734190810848 18446744073709551615 4241488 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 404.16
Current children cumulated vsize (KiB) 2156188

[startup+162.301 s]
/proc/loadavg: 7.92 7.61 7.76 10/186 30676
/proc/meminfo: memFree=11611892/32950940 swapFree=67111528/67111528
[pid=30663] ppid=30661 vsize=2811948 CPUtime=643.73 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) S 30661 30663 30624 0 -1 4202496 645193 0 0 0 62547 1826 0 0 25 0 5 0 601020005 2879434752 640213 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 702987 640213 176 36 0 700892 0
[pid=30663/tid=30669] ppid=30661 vsize=2811948 CPUtime=160.77 cores=1,3,5,7
/proc/30663/task/30669/stat : 30669 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 285435 0 0 0 15990 87 0 0 25 0 5 0 601020153 2879434752 640213 33554432000 4194304 4338100 140734190810848 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=2811948 CPUtime=160.33 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 43666 0 0 0 15171 862 0 0 25 0 5 0 601020153 2879434752 640213 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000418485 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30663/tid=30671] ppid=30661 vsize=2811948 CPUtime=160.57 cores=1,3,5,7
/proc/30663/task/30671/stat : 30671 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 230845 0 0 0 16018 39 0 0 25 0 5 0 601020153 2879434752 640213 33554432000 4194304 4338100 140734190810848 18446744073709551615 215987987832 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30663/tid=30672] ppid=30661 vsize=2811948 CPUtime=160.55 cores=1,3,5,7
/proc/30663/task/30672/stat : 30672 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 47317 0 0 0 15224 831 0 0 25 0 5 0 601020153 2879434752 640213 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000418485 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 643.73
Current children cumulated vsize (KiB) 2811948

[startup+222.301 s]
/proc/loadavg: 8.03 7.70 7.78 9/186 30677
/proc/meminfo: memFree=10221208/32950940 swapFree=67111528/67111528
[pid=30663] ppid=30661 vsize=3496980 CPUtime=883.25 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) S 30661 30663 30624 0 -1 4202496 819870 0 0 0 85779 2546 0 0 25 0 5 0 601020005 3580907520 814890 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 874245 814890 176 36 0 872150 0
[pid=30663/tid=30669] ppid=30661 vsize=3496980 CPUtime=220.72 cores=1,3,5,7
/proc/30663/task/30669/stat : 30669 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 388624 0 0 0 21955 117 0 0 25 0 5 0 601020153 3580907520 814890 33554432000 4194304 4338100 140734190810848 18446744073709551615 4265557 0 0 4096 0 0 0 0 -1 1 0 0 0

################
# More data... #
################

/proc/loadavg: 8.18 8.05 7.91 9/186 30693
/proc/meminfo: memFree=93684/32950940 swapFree=67111524/67111528
[pid=30663] ppid=30661 vsize=8755936 CPUtime=3277.5 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) S 30661 30663 30624 0 -1 4202496 2126683 0 0 0 318303 9447 0 0 25 0 5 0 601020005 8966078464 2121703 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 2188984 2121704 176 36 0 2186889 0
[pid=30663/tid=30669] ppid=30661 vsize=8755936 CPUtime=818.86 cores=1,3,5,7
/proc/30663/task/30669/stat : 30669 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 1188742 0 0 0 81497 389 0 0 25 0 5 0 601020153 8966078464 2121704 33554432000 4194304 4338100 140734190810848 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=8755936 CPUtime=818.41 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 105976 0 0 0 77301 4540 0 0 25 0 5 0 601020153 8966078464 2121704 33554432000 4194304 4338100 140734190810848 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30663/tid=30671] ppid=30661 vsize=8755936 CPUtime=819.67 cores=1,3,5,7
/proc/30663/task/30671/stat : 30671 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 711329 0 0 0 81841 126 0 0 25 0 5 0 601020153 8966078464 2121705 33554432000 4194304 4338100 140734190810848 18446744073709551615 4256749 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30663/tid=30672] ppid=30661 vsize=8755936 CPUtime=819.06 cores=1,3,5,7
/proc/30663/task/30672/stat : 30672 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 82707 0 0 0 77520 4386 0 0 25 0 5 0 601020153 8966078464 2121705 33554432000 4194304 4338100 140734190810848 18446744073709551615 4229521 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3277.5
Current children cumulated vsize (KiB) 8755936

[startup+882.301 s]
/proc/loadavg: 8.18 8.07 7.92 10/186 30695
/proc/meminfo: memFree=94088/32950940 swapFree=67111524/67111528
[pid=30663] ppid=30661 vsize=9675992 CPUtime=3516.67 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) S 30661 30663 30624 0 -1 4202496 2247296 0 0 0 341511 10156 0 0 25 0 5 0 601020005 9908215808 2242316 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 2418998 2242316 176 36 0 2416903 0
[pid=30663/tid=30669] ppid=30661 vsize=9675992 CPUtime=878.55 cores=1,3,5,7
/proc/30663/task/30669/stat : 30669 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 1259143 0 0 0 87438 417 0 0 25 0 5 0 601020153 9908215808 2242316 33554432000 4194304 4338100 140734190810848 18446744073709551615 215988289077 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=9675992 CPUtime=878.26 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 113957 0 0 0 82942 4884 0 0 25 0 5 0 601020153 9908215808 2242316 33554432000 4194304 4338100 140734190810848 18446744073709551615 215987987902 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30663/tid=30671] ppid=30661 vsize=9675992 CPUtime=879.51 cores=1,3,5,7
/proc/30663/task/30671/stat : 30671 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 747942 0 0 0 87817 134 0 0 25 0 5 0 601020153 9908215808 2242316 33554432000 4194304 4338100 140734190810848 18446744073709551615 4284167 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30663/tid=30672] ppid=30661 vsize=9675992 CPUtime=878.86 cores=1,3,5,7
/proc/30663/task/30672/stat : 30672 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 88324 0 0 0 83170 4716 0 0 25 0 5 0 601020153 9908215808 2242316 33554432000 4194304 4338100 140734190810848 18446744073709551615 215987987937 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3516.67
Current children cumulated vsize (KiB) 9675992

[startup+942.301 s]
/proc/loadavg: 8.13 8.07 7.93 10/186 30697
/proc/meminfo: memFree=93984/32950940 swapFree=67111524/67111528
[pid=30663] ppid=30661 vsize=10082632 CPUtime=3756.13 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) S 30661 30663 30624 0 -1 4202496 2358924 0 0 0 364771 10842 0 0 25 0 5 0 601020005 10324615168 2353944 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 2520658 2353944 176 36 0 2518563 0
[pid=30663/tid=30669] ppid=30661 vsize=10082632 CPUtime=938.17 cores=1,3,5,7
/proc/30663/task/30669/stat : 30669 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 1326088 0 0 0 93374 443 0 0 25 0 5 0 601020153 10324615168 2353944 33554432000 4194304 4338100 140734190810848 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=10082632 CPUtime=938.22 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 118012 0 0 0 88602 5220 0 0 25 0 5 0 601020153 10324615168 2353944 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000418485 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30663/tid=30671] ppid=30661 vsize=10082632 CPUtime=939.39 cores=1,3,5,7
/proc/30663/task/30671/stat : 30671 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 782632 0 0 0 93799 140 0 0 25 0 5 0 601020153 10324615168 2353944 33554432000 4194304 4338100 140734190810848 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30663/tid=30672] ppid=30661 vsize=10082632 CPUtime=938.84 cores=1,3,5,7
/proc/30663/task/30672/stat : 30672 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 94262 0 0 0 88851 5033 0 0 25 0 5 0 601020153 10324615168 2353944 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000418485 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3756.13
Current children cumulated vsize (KiB) 10082632

[startup+1002.3 s]
/proc/loadavg: 8.10 8.07 7.93 9/186 30698
/proc/meminfo: memFree=92896/32950940 swapFree=67111524/67111528
[pid=30663] ppid=30661 vsize=10548836 CPUtime=3995.43 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) S 30661 30663 30624 0 -1 4202496 2485885 0 0 0 388022 11521 0 0 25 0 5 0 601020005 10802008064 2480905 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 2637209 2480905 176 36 0 2635114 0
[pid=30663/tid=30669] ppid=30661 vsize=10548836 CPUtime=997.75 cores=1,3,5,7
/proc/30663/task/30669/stat : 30669 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 1404360 0 0 0 99302 473 0 0 25 0 5 0 601020153 10802008064 2480905 33554432000 4194304 4338100 140734190810848 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=10548836 CPUtime=998.15 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 123614 0 0 0 94269 5546 0 0 25 0 5 0 601020153 10802008064 2480905 33554432000 4194304 4338100 140734190810848 18446744073709551615 4240496 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30663/tid=30671] ppid=30661 vsize=10548836 CPUtime=999.26 cores=1,3,5,7
/proc/30663/task/30671/stat : 30671 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 821052 0 0 0 99779 147 0 0 25 0 5 0 601020153 10802008064 2480905 33554432000 4194304 4338100 140734190810848 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30663/tid=30672] ppid=30661 vsize=10548836 CPUtime=998.77 cores=1,3,5,7
/proc/30663/task/30672/stat : 30672 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 98929 0 0 0 94528 5349 0 0 25 0 5 0 601020153 10802008064 2480905 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000418485 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3995.43
Current children cumulated vsize (KiB) 10548836

[startup+1062.3 s]
/proc/loadavg: 8.09 8.07 7.94 9/186 30700
/proc/meminfo: memFree=92632/32950940 swapFree=67111524/67111528
[pid=30663] ppid=30661 vsize=10805332 CPUtime=4234.81 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) S 30661 30663 30624 0 -1 4202496 2565048 0 0 0 411310 12171 0 0 25 0 5 0 601020005 11064659968 2560068 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 2701333 2560068 176 36 0 2699238 0
[pid=30663/tid=30669] ppid=30661 vsize=10805332 CPUtime=1057.63 cores=1,3,5,7
/proc/30663/task/30669/stat : 30669 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 1445348 0 0 0 105268 495 0 0 25 0 5 0 601020153 11064659968 2560069 33554432000 4194304 4338100 140734190810848 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=10805332 CPUtime=1058.1 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 132400 0 0 0 99944 5866 0 0 25 0 5 0 601020153 11064659968 2560069 33554432000 4194304 4338100 140734190810848 18446744073709551615 4240540 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30663/tid=30671] ppid=30661 vsize=10805332 CPUtime=1059.17 cores=1,3,5,7
/proc/30663/task/30671/stat : 30671 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 843966 0 0 0 105765 152 0 0 25 0 5 0 601020153 11064659968 2560069 33554432000 4194304 4338100 140734190810848 18446744073709551615 4230438 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30663/tid=30672] ppid=30661 vsize=10805332 CPUtime=1058.4 cores=1,3,5,7
/proc/30663/task/30672/stat : 30672 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 105405 0 0 0 100188 5652 0 0 25 0 5 0 601020153 11064659968 2560069 33554432000 4194304 4338100 140734190810848 18446744073709551615 4229520 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4234.81
Current children cumulated vsize (KiB) 10805332

[startup+1122.3 s]
/proc/loadavg: 8.12 8.08 7.95 9/186 30701
/proc/meminfo: memFree=95928/32950940 swapFree=67111524/67111528
[pid=30663] ppid=30661 vsize=11421548 CPUtime=4474.13 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) S 30661 30663 30624 0 -1 4202496 2723512 0 0 0 434560 12853 0 0 25 0 5 0 601020005 11695665152 2718532 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 2855387 2718532 176 36 0 2853292 0
[pid=30663/tid=30669] ppid=30661 vsize=11421548 CPUtime=1117.54 cores=1,3,5,7
/proc/30663/task/30669/stat : 30669 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 1541313 0 0 0 111226 528 0 0 25 0 5 0 601020153 11695665152 2718532 33554432000 4194304 4338100 140734190810848 18446744073709551615 4253966 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=11421548 CPUtime=1117.79 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 135355 0 0 0 105591 6188 0 0 25 0 5 0 601020153 11695665152 2718532 33554432000 4194304 4338100 140734190810848 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30663/tid=30671] ppid=30661 vsize=11421548 CPUtime=1119 cores=1,3,5,7
/proc/30663/task/30671/stat : 30671 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 902373 0 0 0 111735 165 0 0 25 0 5 0 601020153 11695665152 2718532 33554432000 4194304 4338100 140734190810848 18446744073709551615 215987987857 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30663/tid=30672] ppid=30661 vsize=11421548 CPUtime=1118.29 cores=1,3,5,7
/proc/30663/task/30672/stat : 30672 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 106541 0 0 0 105864 5965 0 0 25 0 5 0 601020153 11695665152 2718532 33554432000 4194304 4338100 140734190810848 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4474.13
Current children cumulated vsize (KiB) 11421548

[startup+1182.3 s]
/proc/loadavg: 8.09 8.08 7.96 10/186 30703
/proc/meminfo: memFree=93876/32950940 swapFree=67111420/67111528
[pid=30663] ppid=30661 vsize=11784396 CPUtime=4713.34 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) S 30661 30663 30624 0 -1 4202496 2827689 0 0 0 457821 13513 0 0 25 0 5 0 601020005 12067221504 2822709 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 2946099 2822709 176 36 0 2944004 0
[pid=30663/tid=30669] ppid=30661 vsize=11784396 CPUtime=1177.27 cores=1,3,5,7
/proc/30663/task/30669/stat : 30669 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 1607877 0 0 0 117169 558 0 0 25 0 5 0 601020153 12067221504 2822709 33554432000 4194304 4338100 140734190810848 18446744073709551615 216000406787 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=11784396 CPUtime=1177.64 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 141889 0 0 0 111256 6508 0 0 25 0 5 0 601020153 12067221504 2822709 33554432000 4194304 4338100 140734190810848 18446744073709551615 4263517 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30663/tid=30671] ppid=30661 vsize=11784396 CPUtime=1178.92 cores=1,3,5,7
/proc/30663/task/30671/stat : 30671 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 926058 0 0 0 117721 171 0 0 25 0 5 0 601020153 12067221504 2822709 33554432000 4194304 4338100 140734190810848 18446744073709551615 4239459 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30663/tid=30672] ppid=30661 vsize=11784396 CPUtime=1178.01 cores=1,3,5,7
/proc/30663/task/30672/stat : 30672 (ttsth-5-4) R 30661 30663 30624 0 -1 4202560 113935 0 0 0 111530 6271 0 0 25 0 5 0 601020153 12067221504 2822709 33554432000 4194304 4338100 140734190810848 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4713.34
Current children cumulated vsize (KiB) 11784396



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1204.1 s]
/proc/loadavg: 8.06 8.07 7.96 6/183 30704
/proc/meminfo: memFree=617900/32950940 swapFree=67111364/67111528
[pid=30663] ppid=30661 vsize=0 CPUtime=4800.03 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) Z 30661 30663 30624 0 -1 4203548 2884933 0 0 0 466245 13758 0 0 18 0 2 0 601020005 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 0 0 0 0 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=0 CPUtime=1199.34 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4203588 144418 0 0 0 113312 6622 0 0 25 0 2 0 601020153 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.03
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1204.21 s]
/proc/loadavg: 8.06 8.07 7.96 6/183 30704
/proc/meminfo: memFree=617900/32950940 swapFree=67111364/67111528
[pid=30663] ppid=30661 vsize=0 CPUtime=4800.13 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) Z 30661 30663 30624 0 -1 4203548 2884933 0 0 0 466245 13768 0 0 18 0 2 0 601020005 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 0 0 0 0 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=0 CPUtime=1199.44 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4203588 144418 0 0 0 113312 6632 0 0 25 0 2 0 601020153 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.13
Current children cumulated vsize (KiB) 0

[startup+1204.4 s]
/proc/loadavg: 8.06 8.07 7.96 6/184 30705
/proc/meminfo: memFree=8667476/32950940 swapFree=67111364/67111528
[pid=30663] ppid=30661 vsize=0 CPUtime=4800.32 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) Z 30661 30663 30624 0 -1 4203548 2884933 0 0 0 466245 13787 0 0 18 0 2 0 601020005 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 0 0 0 0 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=0 CPUtime=1199.64 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4203588 144418 0 0 0 113312 6652 0 0 25 0 2 0 601020153 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.32
Current children cumulated vsize (KiB) 0

[startup+1204.51 s]
/proc/loadavg: 8.06 8.07 7.96 6/184 30705
/proc/meminfo: memFree=8667476/32950940 swapFree=67111364/67111528
[pid=30663] ppid=30661 vsize=0 CPUtime=4800.43 cores=1,3,5,7
/proc/30663/stat : 30663 (ttsth-5-4) Z 30661 30663 30624 0 -1 4203548 2884933 0 0 0 466245 13798 0 0 18 0 2 0 601020005 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/30663/statm: 0 0 0 0 0 0 0
[pid=30663/tid=30670] ppid=30661 vsize=0 CPUtime=1199.74 cores=1,3,5,7
/proc/30663/task/30670/stat : 30670 (ttsth-5-4) R 30661 30663 30624 0 -1 4203588 144418 0 0 0 113312 6662 0 0 25 0 2 0 601020153 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.43
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1204.55
CPU time (s): 4800.47
CPU user time (s): 4662.45
CPU system time (s): 138.021
CPU usage (%): 398.528
Max. virtual memory (cumulated for all children) (KiB): 11990976

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4662.45
system time used= 138.021
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2884933
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 14898
involuntary context switches= 36491

runsolver used 1.9717 second user time and 4.6073 second system time

The end

Launcher Data 

Begin job on node117 at 2011-04-29 22:11:32
IDJOB=3251484
IDBENCH=88223
IDSOLVER=1654
FILE ID=node117/3251484-1304107892
RUNJOBID= node117-1304107892-30644
PBS_JOBID= 13170668
Free space on /tmp= 71676 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m42-q663-n40-p15-s72490337.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3251484-1304107892/watcher-3251484-1304107892 -o /tmp/evaluation-result-3251484-1304107892/solver-3251484-1304107892 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3251484-1304107892.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= ab8e40d31c79e7a71114e3126a46874f
RANDOM SEED=1413771839

node117.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5267.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950940 kB
MemFree:      17422112 kB
Buffers:       2381736 kB
Cached:       12487460 kB
SwapCached:          0 kB
Active:        8167272 kB
Inactive:      6776960 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950940 kB
LowFree:      17422112 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            5996 kB
Writeback:           0 kB
AnonPages:       74116 kB
Mapped:          15176 kB
Slab:           519096 kB
PageTables:       4540 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586996 kB
Committed_AS:   216300 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71676 MiB
End job on node117 at 2011-04-29 22:31:38