Trace number 3257426

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.55 1203.17

General information on the benchmark

NameCRAFTED/
edgematching/all/em_8_4_5_all.cnf
MD5SUMbcbe82a00b39f246e1dc1a85b2c8371e
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark1.69974
Satisfiable
(Un)Satisfiability was proved
Number of variables2420
Number of clauses62900
Sum of the clauses size155908
Maximum clause length36
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 243724
Number of clauses of size 313968
Number of clauses of size 42628
Number of clauses of size 52460
Number of clauses of size over 5120

Solver Data 

0.02/0.05	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3257426-1304046015.cnf 
0.02/0.05	c Using SATTIMEOUT of 4800
0.02/0.05	c Using SATRAM of 15500
0.02/0.05	c Size of subset node = 32
0.02/0.05	c Variables = 2420
0.02/0.05	c Clauses = 62900
0.02/0.05	c Literals = 155908
0.02/0.05	c Reduced size:
0.02/0.05	c Variables = 2420
0.02/0.05	c Clauses = 62900
0.02/0.05	c Literals = 155908
0.02/0.05	c a2
0.09/0.11	c a1
0.19/0.24	c Independent sub-problems = 1
0.19/0.24	c Sub-problem with 2420 variables and 62900 clauses
0.19/0.24	c using permutation method 4
0.19/0.24	c count = 1
0.19/0.24	c Ordering finished 0.220 seconds
0.19/0.24	c mapped 0.230 seconds
0.19/0.24	c permuted 0.230 seconds
0.19/0.24	c about to sort
0.19/0.25	c sorted
0.29/0.37	c clauses now 62900
0.29/0.37	c init finished 0.270 seconds
0.29/0.37	c tree built 0.350 seconds
0.29/0.37	c 
0.29/0.37	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.29/0.37	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.29/0.37	c ................................ | ..........                    |                                  0.3         0.0             
0.29/0.37	c ................................ | ..........                  1 |                                  0.3         2.9             
0.29/0.37	c ................................ | ..........                  2 |                                  0.4         5.6             
0.29/0.37	c ................................ | ..........                  3 |                                  0.4         8.3             
0.29/0.37	c ................................ | ..........                  4 |                                  0.4        11.1             
0.29/0.37	c ................................ | ..........                  5 |                                  0.4        13.9             
0.29/0.37	c ................................ | ..........               1000 | 26           404                 0.4      2631.6         5249
95.85/24.34	c ................................ | ..........            2692000 | 247056       1828420            96.0     28035.8      9121657
191.97/48.44	c ................................ | ..........            4533000 | 374488       2622803           192.1     23592.2     22273657
287.75/72.48	c ................................ | ..........            5148000 | 400143       2840966           288.1     17869.4     35198022
384.34/96.64	c ................................ | ..........            5468000 | 415956       2989931           384.5     14220.7     45369450
480.15/120.63	c ................................ | *.........            5715000 | 430142       3117316           480.3     11899.6     53346805
575.94/144.68	c ................................ | *.........            5943000 | 443992       3260184           576.3     10312.7     61785143
672.14/168.71	c ................................ | *.........            6199000 | 458694       3397130           672.2      9222.0     68619640
767.97/192.75	c ................................ | *.........            6408000 | 472548       3510174           768.2      8341.9     74678940
863.73/216.80	c ................................ | *.........            6626000 | 485766       3645767           864.1      7667.8     82010691
959.89/240.87	c ................................ | **........            6899000 | 500195       3790211           960.2      7185.0     89508516
1056.06/264.97	c ................................ | **........            7078000 | 510616       3919369          1056.2      6701.6     94686870
1151.84/288.95	c ................................ | **........            7243000 | 523179       4019760          1152.0      6287.3     99419351
1248.02/313.04	c ................................ | **........            7418000 | 534976       4135809          1248.0      5943.8    105073264
1343.83/337.05	c ................................ | **........            7680000 | 549962       4254746          1344.0      5714.2    112860730
1440.45/361.22	c ................................ | ***.......            7882000 | 562781       4380317          1440.6      5471.5    120665442
1536.30/385.26	c ................................ | ***.......            8052000 | 575990       4493081          1536.5      5240.3    127492879
1632.54/409.38	c ................................ | ***.......            8246000 | 588075       4616741          1632.8      5050.1    135060240
1727.90/433.22	c ................................ | ***.......            8394000 | 600217       4719261          1728.0      4857.6    138456200
1824.01/457.31	c ................................ | ***.......            8688000 | 611866       4866343          1824.1      4762.9    139881416
1920.21/481.44	c ................................ | ****......            8951000 | 622992       5016227          1920.4      4661.1    140527503
2016.02/505.42	c ................................ | ****......            9160000 | 634601       5125956          2016.1      4543.4    144241439
2111.80/529.49	c ................................ | ****......            9365000 | 644072       5237545          2112.2      4433.8    148159662
2208.02/553.54	c ................................ | ****......            9567000 | 653598       5361609          2208.2      4332.5    152778822
2304.16/577.61	c ................................ | ****......            9798000 | 664982       5473371          2304.2      4252.2    154494844
2399.92/601.66	c ................................ | *****.....           10054000 | 677974       5591580          2400.2      4188.9    154494844
2495.72/625.67	c ................................ | *****.....           10285000 | 686942       5710710          2496.0      4120.6    154522366
2591.91/649.75	c ................................ | *****.....           10505000 | 698736       5812370          2592.1      4052.7    157162340
2688.11/673.81	c ................................ | *****.....           10698000 | 707451       5902567          2688.2      3979.7    159995762
2783.91/697.84	c ................................ | *****.....           10956000 | 717924       5996859          2784.1      3935.2    164577898
2880.09/721.91	c ................................ | ******....           11212000 | 728462       6088749          2880.1      3892.9    166072740
2976.29/746.01	c ................................ | ******....           11407000 | 736935       6195926          2976.3      3832.6    167245159
3072.08/770.09	c ................................ | ******....           11584000 | 745197       6291615          3072.4      3770.3    170323969
3168.29/794.15	c ................................ | ******....           11747000 | 751758       6390628          3168.5      3707.4    173134929
3264.10/818.19	c ................................ | ******....           11912000 | 758561       6484595          3264.5      3649.0    175941760
3360.24/842.27	c ................................ | *******...           12078000 | 768939       6561506          3360.6      3594.1    176370046
3456.07/866.29	c ................................ | *******...           12234000 | 776462       6635962          3456.4      3539.5    177745633
3551.91/890.22	c ................................ | *******...           12382000 | 783701       6710386          3552.0      3485.9    180810934
3648.15/914.34	c ................................ | *******...           12526000 | 789417       6788761          3648.3      3433.4    184902778
3744.37/938.40	c ................................ | *******...           12724000 | 798447       6869262          3744.4      3398.2    189726745
3840.11/962.42	c ................................ | ********..           13036000 | 810642       6978056          3840.2      3394.6    192853596
3935.91/986.43	c ................................ | ********..           13279000 | 821812       7076808          3936.1      3373.7    192853596
4032.52/1010.62	c ................................ | ********..           13462000 | 830202       7161837          4032.6      3338.3    192853596
4128.30/1034.62	c ................................ | ********..           13675000 | 842530       7246382          4128.4      3312.4    192853596
4223.71/1058.58	c ................................ | ********..           13873000 | 851270       7334381          4224.1      3284.3    192853596
4319.96/1082.63	c ................................ | *********.           14060000 | 859601       7427132          4320.1      3254.6    192853596
4416.19/1106.79	c ................................ | *********.           14240000 | 870002       7507261          4416.5      3224.3    192853596
4511.99/1130.74	c ................................ | *********.           14403000 | 877639       7584727          4512.2      3192.0    192853596
4607.81/1154.76	c ................................ | *********.           14562000 | 883963       7658506          4608.1      3160.1    192863454
4704.03/1178.86	c ................................ | *********.           14709000 | 889615       7728829          4704.3      3126.7    193529770

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3257426-1304046015/watcher-3257426-1304046015 -o /tmp/evaluation-result-3257426-1304046015/solver-3257426-1304046015 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3257426-1304046015.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.75 5.39 6.70 3/177 19201
/proc/meminfo: memFree=11751984/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=9772 CPUtime=0 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) R 19199 19201 18020 0 -1 4202496 491 0 0 0 0 0 0 0 23 0 1 0 594832117 10006528 433 33554432000 4194304 4338100 140733554138640 18446744073709551615 216734875744 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/19201/statm: 2443 433 147 36 0 348 0

[startup+0.030158 s]
/proc/loadavg: 3.75 5.39 6.70 3/177 19201
/proc/meminfo: memFree=11751984/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=14084 CPUtime=0.02 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) R 19199 19201 18020 0 -1 4202496 1555 0 0 0 2 0 0 0 23 0 1 0 594832117 14422016 1497 33554432000 4194304 4338100 140733554138640 18446744073709551615 216734839493 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/19201/statm: 3521 1499 148 36 0 1426 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 14084

[startup+0.10116 s]
/proc/loadavg: 3.75 5.39 6.70 3/177 19201
/proc/meminfo: memFree=11751984/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=24908 CPUtime=0.09 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) R 19199 19201 18020 0 -1 4202496 4259 0 0 0 9 0 0 0 23 0 1 0 594832117 25505792 4200 33554432000 4194304 4338100 140733554138640 18446744073709551615 216734875184 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/19201/statm: 6227 4200 153 36 0 4132 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 24908

[startup+0.301138 s]
/proc/loadavg: 3.75 5.39 6.70 3/177 19201
/proc/meminfo: memFree=11751984/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=52616 CPUtime=0.29 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) R 19199 19201 18020 0 -1 4202496 11582 0 0 0 28 1 0 0 25 0 1 0 594832117 53878784 11154 33554432000 4194304 4338100 140733554138640 18446744073709551615 216734839493 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/19201/statm: 13154 11161 164 36 0 11059 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 52616

[startup+1.29633 s]
/proc/loadavg: 3.75 5.39 6.70 3/177 19201
/proc/meminfo: memFree=11751984/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=267524 CPUtime=6.58 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 21299 0 0 0 653 5 0 0 25 0 5 0 594832117 273944576 20871 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 66881 20871 176 36 0 64786 0
Current children cumulated CPU time (s) 6.58
Current children cumulated vsize (KiB) 267524

[startup+1.97973 s]
/proc/loadavg: 3.75 5.39 6.70 3/177 19201
/proc/meminfo: memFree=11751984/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=267524 CPUtime=6.77 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 21527 0 0 0 672 5 0 0 25 0 5 0 594832117 273944576 21099 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 66881 21099 176 36 0 64786 0
Current children cumulated CPU time (s) 6.77
Current children cumulated vsize (KiB) 267524

[startup+3.10053 s]
/proc/loadavg: 3.61 5.33 6.67 9/186 19210
/proc/meminfo: memFree=11496976/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=280960 CPUtime=11.25 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 26143 0 0 0 1118 7 0 0 25 0 5 0 594832117 287703040 25715 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 70240 25716 176 36 0 68145 0
[pid=19201/tid=19203] ppid=19199 vsize=280960 CPUtime=2.71 cores=0,2,4,6
/proc/19201/task/19203/stat : 19203 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 4278 0 0 0 270 1 0 0 20 0 5 0 594832154 287703040 25716 33554432000 4194304 4338100 140733554138640 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=19201/tid=19204] ppid=19199 vsize=280960 CPUtime=2.71 cores=0,2,4,6
/proc/19201/task/19204/stat : 19204 (ttsth-5-4) S 19199 19201 18020 0 -1 4202560 1766 0 0 0 271 0 0 0 23 0 5 0 594832154 287703040 25716 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746991366 0 0 4096 0 18446604453253970112 0 0 -1 0 0 0 0
[pid=19201/tid=19205] ppid=19199 vsize=280960 CPUtime=2.71 cores=0,2,4,6
/proc/19201/task/19205/stat : 19205 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 1977 0 0 0 270 1 0 0 23 0 5 0 594832154 287703040 25717 33554432000 4194304 4338100 140733554138640 18446744073709551615 4253990 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=19201/tid=19206] ppid=19199 vsize=280960 CPUtime=2.71 cores=0,2,4,6
/proc/19201/task/19206/stat : 19206 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 3343 0 0 0 270 1 0 0 21 0 5 0 594832154 287703040 25717 33554432000 4194304 4338100 140733554138640 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 11.25
Current children cumulated vsize (KiB) 280960

[startup+6.30098 s]
/proc/loadavg: 3.96 5.38 6.68 9/186 19210
/proc/meminfo: memFree=11407572/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=316504 CPUtime=24.04 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 34847 0 0 0 2393 11 0 0 25 0 5 0 594832117 324100096 34419 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 79126 34419 176 36 0 77031 0
[pid=19201/tid=19203] ppid=19199 vsize=316504 CPUtime=5.91 cores=0,2,4,6
/proc/19201/task/19203/stat : 19203 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 6971 0 0 0 588 3 0 0 25 0 5 0 594832154 324100096 34419 33554432000 4194304 4338100 140733554138640 18446744073709551615 4263469 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=19201/tid=19204] ppid=19199 vsize=316504 CPUtime=5.91 cores=0,2,4,6
/proc/19201/task/19204/stat : 19204 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 4303 0 0 0 590 1 0 0 25 0 5 0 594832154 324100096 34419 33554432000 4194304 4338100 140733554138640 18446744073709551615 216734875162 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=19201/tid=19205] ppid=19199 vsize=316504 CPUtime=5.91 cores=0,2,4,6
/proc/19201/task/19205/stat : 19205 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 3717 0 0 0 590 1 0 0 25 0 5 0 594832154 324100096 34419 33554432000 4194304 4338100 140733554138640 18446744073709551615 216734875248 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=19201/tid=19206] ppid=19199 vsize=316504 CPUtime=5.89 cores=0,2,4,6
/proc/19201/task/19206/stat : 19206 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 5076 0 0 0 588 1 0 0 18 0 5 0 594832154 324100096 34419 33554432000 4194304 4338100 140733554138640 18446744073709551615 216734573584 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.04
Current children cumulated vsize (KiB) 316504

[startup+12.7009 s]
/proc/loadavg: 4.29 5.42 6.69 9/186 19211
/proc/meminfo: memFree=11246872/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=376348 CPUtime=49.57 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 52386 0 0 0 4939 18 0 0 25 0 5 0 594832117 385380352 51958 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 94087 51958 176 36 0 91992 0
[pid=19201/tid=19203] ppid=19199 vsize=376348 CPUtime=12.31 cores=0,2,4,6
/proc/19201/task/19203/stat : 19203 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 10006 0 0 0 1226 5 0 0 25 0 5 0 594832154 385380352 51958 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254317 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=19201/tid=19204] ppid=19199 vsize=376348 CPUtime=12.31 cores=0,2,4,6
/proc/19201/task/19204/stat : 19204 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 10827 0 0 0 1227 4 0 0 19 0 5 0 594832154 385380352 51958 33554432000 4194304 4338100 140733554138640 18446744073709551615 216734875306 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=19201/tid=19205] ppid=19199 vsize=376348 CPUtime=12.31 cores=0,2,4,6
/proc/19201/task/19205/stat : 19205 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 5484 0 0 0 1228 3 0 0 25 0 5 0 594832154 385380352 51958 33554432000 4194304 4338100 140733554138640 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=19201/tid=19206] ppid=19199 vsize=376348 CPUtime=12.25 cores=0,2,4,6
/proc/19201/task/19206/stat : 19206 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 11289 0 0 0 1222 3 0 0 18 0 5 0 594832154 385380352 51958 33554432000 4194304 4338100 140733554138640 18446744073709551615 216734573984 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 49.57
Current children cumulated vsize (KiB) 376348

[startup+25.5066 s]
/proc/loadavg: 5.11 5.55 6.71 9/186 19211
/proc/meminfo: memFree=10770592/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=622996 CPUtime=100.65 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 102212 0 0 0 10008 57 0 0 25 0 5 0 594832117 637947904 101784 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 155749 101784 176 36 0 153654 0
[pid=19201/tid=19203] ppid=19199 vsize=622996 CPUtime=25.12 cores=0,2,4,6
/proc/19201/task/19203/stat : 19203 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 14763 0 0 0 2495 17 0 0 25 0 5 0 594832154 637947904 101784 33554432000 4194304 4338100 140733554138640 18446744073709551615 216734574014 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=19201/tid=19204] ppid=19199 vsize=622996 CPUtime=25.12 cores=0,2,4,6
/proc/19201/task/19204/stat : 19204 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 30352 0 0 0 2497 15 0 0 25 0 5 0 594832154 637947904 101784 33554432000 4194304 4338100 140733554138640 18446744073709551615 4279533 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=19201/tid=19205] ppid=19199 vsize=622996 CPUtime=25.11 cores=0,2,4,6
/proc/19201/task/19205/stat : 19205 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 10287 0 0 0 2497 14 0 0 25 0 5 0 594832154 637947904 101784 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254429 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=19201/tid=19206] ppid=19199 vsize=622996 CPUtime=24.9 cores=0,2,4,6
/proc/19201/task/19206/stat : 19206 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 32030 0 0 0 2483 7 0 0 18 0 5 0 594832154 637947904 101784 33554432000 4194304 4338100 140733554138640 18446744073709551615 216734573620 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 100.65
Current children cumulated vsize (KiB) 622996

[startup+51.1012 s]
/proc/loadavg: 6.09 5.74 6.74 9/186 19221
/proc/meminfo: memFree=9941608/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=1081308 CPUtime=202.74 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 219707 0 0 0 20106 168 0 0 25 0 5 0 594832117 1107259392 219279 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 270327 219279 176 36 0 268232 0
[pid=19201/tid=19203] ppid=19199 vsize=1081308 CPUtime=50.7 cores=0,2,4,6
/proc/19201/task/19203/stat : 19203 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 43785 0 0 0 5011 59 0 0 25 0 5 0 594832154 1107259392 219279 33554432000 4194304 4338100 140733554138640 18446744073709551615 216747004597 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=19201/tid=19204] ppid=19199 vsize=1081308 CPUtime=50.71 cores=0,2,4,6
/proc/19201/task/19204/stat : 19204 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 58950 0 0 0 5035 36 0 0 25 0 5 0 594832154 1107259392 219279 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=19201/tid=19205] ppid=19199 vsize=1081308 CPUtime=50.71 cores=0,2,4,6
/proc/19201/task/19205/stat : 19205 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 40897 0 0 0 5016 55 0 0 25 0 5 0 594832154 1107259392 219279 33554432000 4194304 4338100 140733554138640 18446744073709551615 216747004597 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=19201/tid=19206] ppid=19199 vsize=1081308 CPUtime=50.23 cores=0,2,4,6
/proc/19201/task/19206/stat : 19206 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 61295 0 0 0 5008 15 0 0 18 0 5 0 594832154 1107259392 219279 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 202.74
Current children cumulated vsize (KiB) 1081308

[startup+102.3 s]
/proc/loadavg: 7.22 6.10 6.81 9/186 19222
/proc/meminfo: memFree=8711448/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=1829072 CPUtime=407.09 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 407544 0 0 0 40381 328 0 0 25 0 5 0 594832117 1872969728 407116 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 457268 407116 176 36 0 455173 0
[pid=19201/tid=19203] ppid=19199 vsize=1829072 CPUtime=101.9 cores=0,2,4,6
/proc/19201/task/19203/stat : 19203 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 99080 0 0 0 10068 122 0 0 25 0 5 0 594832154 1872969728 407116 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=19201/tid=19204] ppid=19199 vsize=1829072 CPUtime=101.89 cores=0,2,4,6
/proc/19201/task/19204/stat : 19204 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 97447 0 0 0 10123 66 0 0 25 0 5 0 594832154 1872969728 407116 33554432000 4194304 4338100 140733554138640 18446744073709551615 4239479 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=19201/tid=19205] ppid=19199 vsize=1829072 CPUtime=101.9 cores=0,2,4,6
/proc/19201/task/19205/stat : 19205 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 96684 0 0 0 10078 112 0 0 25 0 5 0 594832154 1872969728 407116 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=19201/tid=19206] ppid=19199 vsize=1829072 CPUtime=100.97 cores=0,2,4,6
/proc/19201/task/19206/stat : 19206 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 99553 0 0 0 10074 23 0 0 18 0 5 0 594832154 1872969728 407116 33554432000 4194304 4338100 140733554138640 18446744073709551615 4286296 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 407.09
Current children cumulated vsize (KiB) 1829072

[startup+162.301 s]
/proc/loadavg: 7.79 6.47 6.89 9/186 19224
/proc/meminfo: memFree=7498488/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=2443084 CPUtime=646.59 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 564122 0 0 0 64190 469 0 0 25 0 5 0 594832117 2501718016 563694 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 610771 563694 176 36 0 608676 0
[pid=19201/tid=19203] ppid=19199 vsize=2443084 CPUtime=161.91 cores=0,2,4,6
/proc/19201/task/19203/stat : 19203 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 144312 0 0 0 16015 176 0 0 25 0 5 0 594832154 2501718016 563694 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=19201/tid=19204] ppid=19199 vsize=2443084 CPUtime=161.9 cores=0,2,4,6
/proc/19201/task/19204/stat : 19204 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 131530 0 0 0 16095 95 0 0 25 0 5 0 594832154 2501718016 563694 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254221 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=19201/tid=19205] ppid=19199 vsize=2443084 CPUtime=161.9 cores=0,2,4,6
/proc/19201/task/19205/stat : 19205 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 141605 0 0 0 16029 161 0 0 25 0 5 0 594832154 2501718016 563694 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=19201/tid=19206] ppid=19199 vsize=2443084 CPUtime=160.49 cores=0,2,4,6
/proc/19201/task/19206/stat : 19206 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 131895 0 0 0 16015 34 0 0 19 0 5 0 594832154 2501718016 563694 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 646.59
Current children cumulated vsize (KiB) 2443084

[startup+222.3 s]
/proc/loadavg: 8.06 6.79 6.97 9/186 19225
/proc/meminfo: memFree=6368836/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=2980744 CPUtime=886.06 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 699748 0 0 0 87996 610 0 0 25 0 5 0 594832117 3052281856 699320 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 745186 699321 176 36 0 743091 0
[pid=19201/tid=19203] ppid=19199 vsize=2980744 CPUtime=221.89 cores=0,2,4,6
/proc/19201/task/19203/stat : 19203 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 179669 0 0 0 21960 229 0 0 25 0 5 0 594832154 3052281856 699321 33554432000 4194304 4338100 140733554138640 18446744073709551615 216747004597 0 0 4096 0 0 0 0 -1 4 0 0 0

################
# More data... #
################

/proc/meminfo: memFree=902312/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=5944548 CPUtime=3041.34 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 1423948 0 0 0 302510 1624 0 0 25 0 5 0 594832117 6087217152 1423520 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 1486137 1423520 176 36 0 1484042 0
[pid=19201/tid=19203] ppid=19199 vsize=5944548 CPUtime=760.5 cores=0,2,4,6
/proc/19201/task/19203/stat : 19203 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 355858 0 0 0 75703 347 0 0 18 0 5 0 594832154 6087217152 1423520 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254451 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=19201/tid=19204] ppid=19199 vsize=5944548 CPUtime=759.97 cores=0,2,4,6
/proc/19201/task/19204/stat : 19204 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 358666 0 0 0 75563 434 0 0 25 0 5 0 594832154 6087217152 1423520 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254451 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=19201/tid=19205] ppid=19199 vsize=5944548 CPUtime=761.43 cores=0,2,4,6
/proc/19201/task/19205/stat : 19205 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 370603 0 0 0 75685 458 0 0 25 0 5 0 594832154 6087217152 1423520 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254239 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=19201/tid=19206] ppid=19199 vsize=5944548 CPUtime=759.03 cores=0,2,4,6
/proc/19201/task/19206/stat : 19206 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 324041 0 0 0 75522 381 0 0 25 0 5 0 594832154 6087217152 1423520 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3041.34
Current children cumulated vsize (KiB) 5944548

[startup+822.301 s]
/proc/loadavg: 8.06 7.93 7.50 9/186 19241
/proc/meminfo: memFree=593436/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=6165092 CPUtime=3280.8 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 1479812 0 0 0 326369 1711 0 0 25 0 5 0 594832117 6313054208 1479384 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 1541273 1479384 176 36 0 1539178 0
[pid=19201/tid=19203] ppid=19199 vsize=6165092 CPUtime=820.35 cores=0,2,4,6
/proc/19201/task/19203/stat : 19203 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 370065 0 0 0 81682 353 0 0 20 0 5 0 594832154 6313054208 1479384 33554432000 4194304 4338100 140733554138640 18446744073709551615 216747004597 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=19201/tid=19204] ppid=19199 vsize=6165092 CPUtime=819.81 cores=0,2,4,6
/proc/19201/task/19204/stat : 19204 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 373006 0 0 0 81515 466 0 0 25 0 5 0 594832154 6313054208 1479384 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=19201/tid=19205] ppid=19199 vsize=6165092 CPUtime=821.43 cores=0,2,4,6
/proc/19201/task/19205/stat : 19205 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 384381 0 0 0 81665 478 0 0 25 0 5 0 594832154 6313054208 1479384 33554432000 4194304 4338100 140733554138640 18446744073709551615 216747004597 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=19201/tid=19206] ppid=19199 vsize=6165092 CPUtime=818.8 cores=0,2,4,6
/proc/19201/task/19206/stat : 19206 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 337580 0 0 0 81470 410 0 0 25 0 5 0 594832154 6313054208 1479384 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3280.8
Current children cumulated vsize (KiB) 6165092

[startup+882.3 s]
/proc/loadavg: 8.13 7.98 7.55 9/186 19243
/proc/meminfo: memFree=253692/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=6269636 CPUtime=3520.38 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 1506937 0 0 0 350241 1797 0 0 25 0 5 0 594832117 6420107264 1506509 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 1567409 1506509 176 36 0 1565314 0
[pid=19201/tid=19203] ppid=19199 vsize=6269636 CPUtime=880.25 cores=0,2,4,6
/proc/19201/task/19203/stat : 19203 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 374568 0 0 0 87666 359 0 0 18 0 5 0 594832154 6420107264 1506509 33554432000 4194304 4338100 140733554138640 18446744073709551615 216747004597 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=19201/tid=19204] ppid=19199 vsize=6269636 CPUtime=879.81 cores=0,2,4,6
/proc/19201/task/19204/stat : 19204 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 373260 0 0 0 87487 494 0 0 25 0 5 0 594832154 6420107264 1506509 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=19201/tid=19205] ppid=19199 vsize=6269636 CPUtime=881.43 cores=0,2,4,6
/proc/19201/task/19205/stat : 19205 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 401391 0 0 0 87643 500 0 0 25 0 5 0 594832154 6420107264 1506509 33554432000 4194304 4338100 140733554138640 18446744073709551615 216747004597 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=19201/tid=19206] ppid=19199 vsize=6269636 CPUtime=878.48 cores=0,2,4,6
/proc/19201/task/19206/stat : 19206 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 342938 0 0 0 87408 440 0 0 25 0 5 0 594832154 6420107264 1506509 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254362 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3520.38
Current children cumulated vsize (KiB) 6269636

[startup+942.301 s]
/proc/loadavg: 8.16 8.03 7.59 9/186 19244
/proc/meminfo: memFree=93336/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=6608912 CPUtime=3759.91 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 1593588 0 0 0 374111 1880 0 0 25 0 5 0 594832117 6767525888 1593160 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 1652228 1593160 176 36 0 1650133 0
[pid=19201/tid=19203] ppid=19199 vsize=6608912 CPUtime=940.13 cores=0,2,4,6
/proc/19201/task/19203/stat : 19203 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 398008 0 0 0 93646 367 0 0 18 0 5 0 594832154 6767525888 1593160 33554432000 4194304 4338100 140733554138640 18446744073709551615 216747004597 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=19201/tid=19204] ppid=19199 vsize=6608912 CPUtime=939.8 cores=0,2,4,6
/proc/19201/task/19204/stat : 19204 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 394055 0 0 0 93460 520 0 0 25 0 5 0 594832154 6767525888 1593160 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=19201/tid=19205] ppid=19199 vsize=6608912 CPUtime=941.41 cores=0,2,4,6
/proc/19201/task/19205/stat : 19205 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 424250 0 0 0 93619 522 0 0 25 0 5 0 594832154 6767525888 1593160 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=19201/tid=19206] ppid=19199 vsize=6608912 CPUtime=938.17 cores=0,2,4,6
/proc/19201/task/19206/stat : 19206 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 362495 0 0 0 93350 467 0 0 25 0 5 0 594832154 6767525888 1593160 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3759.91
Current children cumulated vsize (KiB) 6608912

[startup+1002.31 s]
/proc/loadavg: 8.21 8.07 7.63 9/186 19246
/proc/meminfo: memFree=96672/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=6730184 CPUtime=3999.4 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 1623753 0 0 0 397975 1965 0 0 25 0 5 0 594832117 6891708416 1623325 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 1682546 1623325 176 36 0 1680451 0
[pid=19201/tid=19203] ppid=19199 vsize=6730184 CPUtime=999.92 cores=0,2,4,6
/proc/19201/task/19203/stat : 19203 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 404786 0 0 0 99618 374 0 0 18 0 5 0 594832154 6891708416 1623325 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=19201/tid=19204] ppid=19199 vsize=6730184 CPUtime=999.8 cores=0,2,4,6
/proc/19201/task/19204/stat : 19204 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 403740 0 0 0 99434 546 0 0 25 0 5 0 594832154 6891708416 1623325 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=19201/tid=19205] ppid=19199 vsize=6730184 CPUtime=1001.4 cores=0,2,4,6
/proc/19201/task/19205/stat : 19205 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 430140 0 0 0 99594 546 0 0 25 0 5 0 594832154 6891708416 1623325 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=19201/tid=19206] ppid=19199 vsize=6730184 CPUtime=997.87 cores=0,2,4,6
/proc/19201/task/19206/stat : 19206 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 370307 0 0 0 99292 495 0 0 25 0 5 0 594832154 6891708416 1623325 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3999.4
Current children cumulated vsize (KiB) 6730184

[startup+1062.3 s]
/proc/loadavg: 8.15 8.07 7.65 9/186 19248
/proc/meminfo: memFree=95472/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=6731240 CPUtime=4238.89 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 1641466 0 0 0 421856 2033 0 0 25 0 5 0 594832117 6892789760 1641038 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 1682810 1641038 176 36 0 1680715 0
[pid=19201/tid=19203] ppid=19199 vsize=6731240 CPUtime=1059.74 cores=0,2,4,6
/proc/19201/task/19203/stat : 19203 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 411770 0 0 0 105596 378 0 0 18 0 5 0 594832154 6892789760 1641038 33554432000 4194304 4338100 140733554138640 18446744073709551615 216747004597 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=19201/tid=19204] ppid=19199 vsize=6731240 CPUtime=1059.81 cores=0,2,4,6
/proc/19201/task/19204/stat : 19204 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 407116 0 0 0 105411 570 0 0 25 0 5 0 594832154 6892789760 1641038 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=19201/tid=19205] ppid=19199 vsize=6731240 CPUtime=1061.39 cores=0,2,4,6
/proc/19201/task/19205/stat : 19205 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 434716 0 0 0 105575 564 0 0 25 0 5 0 594832154 6892789760 1641038 33554432000 4194304 4338100 140733554138640 18446744073709551615 216747004597 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=19201/tid=19206] ppid=19199 vsize=6731240 CPUtime=1057.56 cores=0,2,4,6
/proc/19201/task/19206/stat : 19206 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 373084 0 0 0 105238 518 0 0 25 0 5 0 594832154 6892789760 1641038 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4238.89
Current children cumulated vsize (KiB) 6731240

[startup+1122.3 s]
/proc/loadavg: 8.12 8.07 7.67 10/186 19249
/proc/meminfo: memFree=99820/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=6732428 CPUtime=4478.43 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 1642989 0 0 0 445746 2097 0 0 25 0 5 0 594832117 6894006272 1642561 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 1683107 1642561 176 36 0 1681012 0
[pid=19201/tid=19203] ppid=19199 vsize=6732428 CPUtime=1119.59 cores=0,2,4,6
/proc/19201/task/19203/stat : 19203 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 412241 0 0 0 111577 382 0 0 18 0 5 0 594832154 6894006272 1642561 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254352 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=19201/tid=19204] ppid=19199 vsize=6732428 CPUtime=1119.81 cores=0,2,4,6
/proc/19201/task/19204/stat : 19204 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 407407 0 0 0 111389 592 0 0 25 0 5 0 594832154 6894006272 1642561 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=19201/tid=19205] ppid=19199 vsize=6732428 CPUtime=1121.39 cores=0,2,4,6
/proc/19201/task/19205/stat : 19205 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 435187 0 0 0 111559 580 0 0 25 0 5 0 594832154 6894006272 1642561 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=19201/tid=19206] ppid=19199 vsize=6732428 CPUtime=1117.27 cores=0,2,4,6
/proc/19201/task/19206/stat : 19206 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 373374 0 0 0 111186 541 0 0 25 0 5 0 594832154 6894006272 1642561 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4478.43
Current children cumulated vsize (KiB) 6732428

[startup+1182.3 s]
/proc/loadavg: 8.14 8.08 7.70 9/186 19251
/proc/meminfo: memFree=95612/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=6755644 CPUtime=4718.01 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 1650283 0 0 0 469638 2163 0 0 25 0 5 0 594832117 6917779456 1649855 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 1688911 1649855 176 36 0 1686816 0
[pid=19201/tid=19203] ppid=19199 vsize=6755644 CPUtime=1179.45 cores=0,2,4,6
/proc/19201/task/19203/stat : 19203 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 412605 0 0 0 117560 385 0 0 18 0 5 0 594832154 6917779456 1649855 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254451 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=19201/tid=19204] ppid=19199 vsize=6755644 CPUtime=1179.81 cores=0,2,4,6
/proc/19201/task/19204/stat : 19204 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 407706 0 0 0 117365 616 0 0 25 0 5 0 594832154 6917779456 1649855 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=19201/tid=19205] ppid=19199 vsize=6755644 CPUtime=1181.38 cores=0,2,4,6
/proc/19201/task/19205/stat : 19205 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 441524 0 0 0 117544 594 0 0 25 0 5 0 594832154 6917779456 1649855 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=19201/tid=19206] ppid=19199 vsize=6755644 CPUtime=1176.98 cores=0,2,4,6
/proc/19201/task/19206/stat : 19206 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 373668 0 0 0 117133 565 0 0 25 0 5 0 594832154 6917779456 1649855 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4718.01
Current children cumulated vsize (KiB) 6755644



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.9 s]
/proc/loadavg: 8.10 8.08 7.71 6/184 19252
/proc/meminfo: memFree=1337292/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=6812476 CPUtime=4800.27 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) S 19199 19201 18020 0 -1 4202496 1664687 0 0 0 477837 2190 0 0 25 0 5 0 594832117 6975975424 1664259 33554432000 4194304 4338100 140733554138640 18446744073709551615 216746981349 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 1703119 1664259 176 36 0 1701024 0
[pid=19201/tid=19203] ppid=19199 vsize=6812476 CPUtime=1200.01 cores=0,2,4,6
/proc/19201/task/19203/stat : 19203 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 414432 0 0 0 119615 386 0 0 18 0 5 0 594832154 6975975424 1664259 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254221 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=19201/tid=19204] ppid=19199 vsize=6812476 CPUtime=1200.41 cores=0,2,4,6
/proc/19201/task/19204/stat : 19204 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 414888 0 0 0 119414 627 0 0 25 0 5 0 594832154 6975975424 1664259 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254384 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=19201/tid=19205] ppid=19199 vsize=6812476 CPUtime=1201.98 cores=0,2,4,6
/proc/19201/task/19205/stat : 19205 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 446779 0 0 0 119597 601 0 0 25 0 5 0 594832154 6975975424 1664259 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=19201/tid=19206] ppid=19199 vsize=6812476 CPUtime=1197.47 cores=0,2,4,6
/proc/19201/task/19206/stat : 19206 (ttsth-5-4) R 19199 19201 18020 0 -1 4202560 373808 0 0 0 119174 573 0 0 25 0 5 0 594832154 6975975424 1664259 33554432000 4194304 4338100 140733554138640 18446744073709551615 4254352 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.27
Current children cumulated vsize (KiB) 6812476

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.1 s]
/proc/loadavg: 8.10 8.08 7.71 6/184 19252
/proc/meminfo: memFree=1337292/32951124 swapFree=67102140/67111528
[pid=19201] ppid=19199 vsize=0 CPUtime=4800.48 cores=0,2,4,6
/proc/19201/stat : 19201 (ttsth-5-4) Z 19199 19201 18020 0 -1 4203548 1664707 0 0 0 477839 2209 0 0 18 0 2 0 594832117 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/19201/statm: 0 0 0 0 0 0 0
[pid=19201/tid=19206] ppid=19199 vsize=0 CPUtime=1197.66 cores=0,2,4,6
/proc/19201/task/19206/stat : 19206 (ttsth-5-4) R 19199 19201 18020 0 -1 4203588 373808 0 0 0 119174 592 0 0 25 0 2 0 594832154 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.48
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.17
CPU time (s): 4800.55
CPU user time (s): 4778.4
CPU system time (s): 22.1586
CPU usage (%): 398.994
Max. virtual memory (cumulated for all children) (KiB): 6812476

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4778.4
system time used= 22.1586
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1664707
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 11657
involuntary context switches= 18959

runsolver used 1.92271 second user time and 4.16637 second system time

The end

Launcher Data 

Begin job on node119 at 2011-04-29 05:00:15
IDJOB=3257426
IDBENCH=70922
IDSOLVER=1654
FILE ID=node119/3257426-1304046015
RUNJOBID= node119-1304046014-19181
PBS_JOBID= 13170378
Free space on /tmp= 71812 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT09/CRAFTED/edgematching/all/em_8_4_5_all.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3257426-1304046015/watcher-3257426-1304046015 -o /tmp/evaluation-result-3257426-1304046015/solver-3257426-1304046015 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3257426-1304046015.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= bcbe82a00b39f246e1dc1a85b2c8371e
RANDOM SEED=1813970973

node119.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      11752636 kB
Buffers:       2243992 kB
Cached:       18306212 kB
SwapCached:       5504 kB
Active:        9534388 kB
Inactive:     11080836 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      11752636 kB
SwapTotal:    67111528 kB
SwapFree:     67102140 kB
Dirty:            3176 kB
Writeback:           0 kB
AnonPages:       61648 kB
Mapped:          14652 kB
Slab:           520328 kB
PageTables:       4728 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   211920 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71812 MiB
End job on node119 at 2011-04-29 05:20:20