Trace number 3262882

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.35 1228.12

General information on the benchmark

NameSAT03/handmade/anton/UNSAT/
SGI_30_70_19_80_8-log.shuffled-as.sat03-143.cnf
MD5SUM3aff0c2163848e3eed6bc22fc2c30cdf
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables95
Number of clauses36888
Sum of the clauses size368690
Maximum clause length10
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 538
Number of clauses of size over 536850

Solver Data 

0.07/0.08	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262882-1304095027.cnf 
0.07/0.08	c Using SATTIMEOUT of 4800
0.07/0.08	c Using SATRAM of 15500
0.07/0.08	c Size of subset node = 32
0.07/0.08	c Variables = 95
0.07/0.08	c Clauses = 36888
0.07/0.08	c Literals = 368690
0.07/0.08	c Reduced size:
0.07/0.08	c Variables = 95
0.07/0.08	c Clauses = 36888
0.07/0.08	c Literals = 368690
0.07/0.08	c a2
0.59/0.64	c a1
1.69/1.75	c Independent sub-problems = 1
1.69/1.75	c Sub-problem with 95 variables and 36888 clauses
1.69/1.75	c using permutation method 4
1.69/1.75	c count = 1
1.69/1.75	c Ordering finished 1.720 seconds
1.69/1.75	c mapped 1.740 seconds
1.69/1.75	c permuted 1.740 seconds
1.69/1.75	c about to sort
1.69/1.76	c sorted
1.79/1.84	c clauses now 36888
1.79/1.84	c init finished 1.760 seconds
1.79/1.84	c tree built 1.830 seconds
1.79/1.84	c 
1.79/1.84	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
1.79/1.84	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
1.79/1.84	c ................................ | ..........                    |                                  1.8         0.0             
1.79/1.84	c ................................ | ..........                  1 |                                  1.8         0.5             
1.79/1.84	c ................................ | ..........                  2 |                                  1.8         1.1             
1.79/1.84	c ................................ | ..........                  3 |                                  1.8         1.6             
1.79/1.84	c ................................ | ..........                  4 |                                  1.8         2.2             
1.79/1.84	c ................................ | ..........                  5 |                                  1.8         2.7             
1.79/1.84	c ................................ | ..........               1000 | 197          1169                1.8       543.5         9659
95.87/25.73	c ................................ | ..........            2392000 | 614198       3317253            96.0     24914.1     24833028
191.84/50.15	c ................................ | ..........            4256000 | 1102975      5867856           192.1     22160.9     34101231
287.85/74.44	c ................................ | ..........            5951000 | 1557574      8257282           288.0     20660.3     43339440
384.04/98.90	c ................................ | ..........            7543000 | 1978796      10492046          384.0     19641.2     48841916
479.89/123.43	c ................................ | *.........            9122000 | 2379686      12637244          480.0     19003.4     55616698
575.88/147.94	c ................................ | *.........           10573000 | 2751935      14694436          576.0     18354.6     58255706
671.75/172.36	c ................................ | *.........           12000000 | 3116678      16709089          672.0     17856.9     62435772
767.84/196.85	c ................................ | *.........           13438000 | 3486051      18687884          768.1     17496.0     66126975
863.92/221.32	c ................................ | *.........           14779000 | 3829693      20675657          864.0     17104.7     68401039
959.98/245.82	c ................................ | **........           16212000 | 4204774      22631905          960.1     16886.4     71073313
1055.67/270.29	c ................................ | **........           17611000 | 4568755      24636781         1056.0     16676.3     72283531
1151.78/294.96	c ................................ | **........           19017000 | 4940824      26562839         1152.0     16507.4     75023061
1247.70/319.39	c ................................ | **........           20331000 | 5280070      28445029         1248.0     16290.7     75982924
1344.00/343.91	c ................................ | **........           21651000 | 5614757      30329461         1344.1     16108.7     76968023
1439.70/368.49	c ................................ | ***.......           23038000 | 5966731      32204149         1440.0     15998.1     79811163
1535.89/393.04	c ................................ | ***.......           24458000 | 6325783      34064005         1536.0     15922.8     82294909
1631.78/417.67	c ................................ | ***.......           25842000 | 6680139      35967719         1632.0     15834.3     84570711
1727.77/442.16	c ................................ | ***.......           27187000 | 7029274      37839166         1728.0     15733.2     85896820
1824.04/466.60	c ................................ | ***.......           28538000 | 7380817      39705667         1824.1     15645.3     87877771
1919.78/491.15	c ................................ | ****......           29914000 | 7737317      41590944         1920.0     15580.1     90682397
2015.85/515.53	c ................................ | ****......           31236000 | 8078869      43452133         2016.0     15494.0     92320709
2111.75/540.10	c ................................ | ****......           32473000 | 8397217      45225132         2112.0     15375.2     94571171
2207.91/564.73	c ................................ | ****......           33668000 | 8701902      46932254         2208.0     15248.0     95320250
2303.71/589.18	c ................................ | ****......           35012000 | 9044789      48706456         2304.1     15195.8     96718103
2399.83/613.74	c ................................ | *****.....           36265000 | 9364251      50550131         2400.0     15110.4     98989997
2495.87/638.25	c ................................ | *****.....           37591000 | 9705209      52367299         2496.1     15060.0    101645727
2591.83/662.75	c ................................ | *****.....           38873000 | 10035681     54122760         2592.0     14997.1    103266764
2688.06/687.30	c ................................ | *****.....           40080000 | 10348223     55904143         2688.1     14910.3    104097467
2784.02/711.90	c ................................ | *****.....           41314000 | 10670006     57693465         2784.0     14839.7    105776883
2880.02/736.51	c ................................ | ******....           42461000 | 10969322     59325318         2880.1     14742.9    108244559
2975.84/761.14	c ................................ | ******....           43589000 | 11266646     60949596         2976.0     14646.7    108571592
3071.96/785.61	c ................................ | ******....           44846000 | 11591170     62686950         3072.0     14598.3    109925281
3167.78/810.27	c ................................ | ******....           46115000 | 11918061     64404482         3168.1     14556.3    112249744
3260.37/833.98	c *............................... | ******....           47268568 | 12217105     66036999         3260.7     14496.4    114185512
3263.91/834.84	c *............................... | ******....           47322000 | 12230799     66103672         3264.1     14497.9    114185512
3359.72/859.48	c *............................... | *******...           48634000 | 12569789     67859439         3360.0     14474.3    116577902
3455.68/884.18	c *............................... | *******...           49878000 | 12893501     69572134         3456.0     14432.2    118484494
3551.96/908.71	c *............................... | *******...           51156000 | 13220016     71308662         3552.0     14401.9    120376518
3647.89/933.33	c *............................... | *******...           52374000 | 13536412     72957010         3648.0     14356.8    121603367
3743.70/957.99	c *............................... | *******...           53651000 | 13876925     74646681         3744.0     14329.7    123401349
3839.93/982.52	c *............................... | ********..           54808000 | 14185280     76289119         3840.0     14272.8    125022726
3936.36/1007.12	c *............................... | ********..           56060000 | 14516056     77989314         3936.4     14241.3    126957374
4031.75/1031.57	c *............................... | ********..           57184000 | 14809766     79586039         4032.0     14182.4    128876491
4127.70/1056.08	c *............................... | ********..           58280000 | 15090883     81198663         4128.0     14118.1    130133838
4223.63/1080.60	c *............................... | ********..           59444000 | 15391582     82782103         4224.0     14072.9    132164361
4319.65/1105.29	c *............................... | *********.           60506000 | 15663835     84351015         4320.0     14006.0    133504278
4415.90/1129.83	c *............................... | *********.           61609000 | 15948477     85854236         4416.0     13951.2    135251004
4511.71/1154.28	c *............................... | *********.           62770000 | 16248706     87479794         4512.0     13911.7    136462414
4608.01/1178.70	c *............................... | *********.           63824000 | 16519629     89056490         4608.0     13850.6    137500049
4703.77/1203.16	c *............................... | *********.           64888000 | 16796443     90583924         4704.0     13794.1    138000174
4799.68/1227.78	c CPU limit (4800) reached
4799.68/1227.78	c Time = 4800.00 seconds
4799.68/1227.78	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3262882-1304095027/watcher-3262882-1304095027 -o /tmp/evaluation-result-3262882-1304095027/solver-3262882-1304095027 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262882-1304095027.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.34 7.70 7.88 2/177 905
/proc/meminfo: memFree=30878044/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=8840 CPUtime=0 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) R 903 905 32461 0 -1 4202496 277 0 0 0 0 0 0 0 22 0 1 0 599733385 9052160 219 33554432000 4194304 4338100 140735006639168 18446744073709551615 218880225989 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/905/statm: 2210 220 146 36 0 115 0

[startup+0.074378 s]
/proc/loadavg: 6.34 7.70 7.88 2/177 905
/proc/meminfo: memFree=30878044/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=14048 CPUtime=0.07 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) R 903 905 32461 0 -1 4202496 1591 0 0 0 7 0 0 0 22 0 1 0 599733385 14385152 1533 33554432000 4194304 4338100 140735006639168 18446744073709551615 218880225989 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/905/statm: 3545 1533 148 36 0 1450 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 14048

[startup+0.100379 s]
/proc/loadavg: 6.34 7.70 7.88 2/177 905
/proc/meminfo: memFree=30878044/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=26020 CPUtime=0.09 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) R 903 905 32461 0 -1 4202496 3965 0 0 0 9 0 0 0 22 0 1 0 599733385 26644480 3782 33554432000 4194304 4338100 140735006639168 18446744073709551615 4204238 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/905/statm: 6505 3785 416 36 0 4410 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 26020

[startup+0.300339 s]
/proc/loadavg: 6.34 7.70 7.88 2/177 905
/proc/meminfo: memFree=30878044/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=62360 CPUtime=0.29 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) R 903 905 32461 0 -1 4202496 7407 0 0 0 28 1 0 0 24 0 1 0 599733385 63856640 7030 33554432000 4194304 4338100 140735006639168 18446744073709551615 4206976 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/905/statm: 15590 7030 283 36 0 13495 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 62360

[startup+0.700223 s]
/proc/loadavg: 6.34 7.70 7.88 2/177 905
/proc/meminfo: memFree=30878044/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=18948 CPUtime=0.69 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) R 903 905 32461 0 -1 4202496 7432 0 0 0 68 1 0 0 25 0 1 0 599733385 19402752 2776 33554432000 4194304 4338100 140735006639168 18446744073709551615 4296773 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/905/statm: 4737 2776 224 36 0 2642 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 18948

[startup+1.50108 s]
/proc/loadavg: 6.34 7.70 7.88 6/182 911
/proc/meminfo: memFree=30814152/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=20136 CPUtime=1.49 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) R 903 905 32461 0 -1 4202496 7764 0 0 0 148 1 0 0 25 0 1 0 599733385 20619264 3043 33554432000 4194304 4338100 140735006639168 18446744073709551615 4245192 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/905/statm: 5034 3043 159 36 0 2939 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 20136

[startup+3.10074 s]
/proc/loadavg: 6.34 7.70 7.88 10/186 915
/proc/meminfo: memFree=30765016/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=375352 CPUtime=6.83 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) S 903 905 32461 0 -1 4202496 37064 0 0 0 675 8 0 0 25 0 5 0 599733385 384360448 32338 33554432000 4194304 4338100 140735006639168 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 93838 32338 176 36 0 91743 0
[pid=905/tid=912] ppid=903 vsize=375352 CPUtime=1.23 cores=0,2,4,6
/proc/905/task/912/stat : 912 (ttsth-5-4) R 903 905 32461 0 -1 4202560 7126 0 0 0 122 1 0 0 19 0 5 0 599733569 384360448 32338 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254321 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=905/tid=913] ppid=903 vsize=375352 CPUtime=1.24 cores=0,2,4,6
/proc/905/task/913/stat : 913 (ttsth-5-4) R 903 905 32461 0 -1 4202560 7244 0 0 0 123 1 0 0 18 0 5 0 599733569 384360448 32338 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254400 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=905/tid=914] ppid=903 vsize=375352 CPUtime=1.24 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4202560 7460 0 0 0 123 1 0 0 19 0 5 0 599733569 384360448 32338 33554432000 4194304 4338100 140735006639168 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=905/tid=915] ppid=903 vsize=375352 CPUtime=1.25 cores=0,2,4,6
/proc/905/task/915/stat : 915 (ttsth-5-4) R 903 905 32461 0 -1 4202560 6991 0 0 0 123 2 0 0 18 0 5 0 599733569 384360448 32339 33554432000 4194304 4338100 140735006639168 18446744073709551615 218892379375 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 6.83
Current children cumulated vsize (KiB) 375352

[startup+6.30118 s]
/proc/loadavg: 6.55 7.72 7.89 10/186 915
/proc/meminfo: memFree=30481520/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=552016 CPUtime=19.49 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) S 903 905 32461 0 -1 4202496 84455 0 0 0 1930 19 0 0 25 0 5 0 599733385 565264384 79729 33554432000 4194304 4338100 140735006639168 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 138004 79729 176 36 0 135909 0
[pid=905/tid=912] ppid=903 vsize=552016 CPUtime=4.37 cores=0,2,4,6
/proc/905/task/912/stat : 912 (ttsth-5-4) R 903 905 32461 0 -1 4202560 19015 0 0 0 433 4 0 0 20 0 5 0 599733569 565264384 79729 33554432000 4194304 4338100 140735006639168 18446744073709551615 4253954 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=905/tid=913] ppid=903 vsize=552016 CPUtime=4.41 cores=0,2,4,6
/proc/905/task/913/stat : 913 (ttsth-5-4) R 903 905 32461 0 -1 4202560 18678 0 0 0 438 3 0 0 25 0 5 0 599733569 565264384 79729 33554432000 4194304 4338100 140735006639168 18446744073709551615 218892379395 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=905/tid=914] ppid=903 vsize=552016 CPUtime=4.41 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4202560 19516 0 0 0 438 3 0 0 25 0 5 0 599733569 565264384 79729 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=905/tid=915] ppid=903 vsize=552016 CPUtime=4.41 cores=0,2,4,6
/proc/905/task/915/stat : 915 (ttsth-5-4) R 903 905 32461 0 -1 4202560 19002 0 0 0 436 5 0 0 25 0 5 0 599733569 565264384 79729 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 19.49
Current children cumulated vsize (KiB) 552016

[startup+12.7011 s]
/proc/loadavg: 6.67 7.73 7.89 10/186 915
/proc/meminfo: memFree=30135476/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=752228 CPUtime=44.75 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) S 903 905 32461 0 -1 4202496 138609 0 0 0 4442 33 0 0 25 0 5 0 599733385 770281472 133883 33554432000 4194304 4338100 140735006639168 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 188057 133883 176 36 0 185962 0
[pid=905/tid=912] ppid=903 vsize=752228 CPUtime=10.65 cores=0,2,4,6
/proc/905/task/912/stat : 912 (ttsth-5-4) R 903 905 32461 0 -1 4202560 33826 0 0 0 1056 9 0 0 18 0 5 0 599733569 770281472 133883 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=905/tid=913] ppid=903 vsize=752228 CPUtime=10.74 cores=0,2,4,6
/proc/905/task/913/stat : 913 (ttsth-5-4) R 903 905 32461 0 -1 4202560 31182 0 0 0 1068 6 0 0 25 0 5 0 599733569 770281472 133883 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=905/tid=914] ppid=903 vsize=752228 CPUtime=10.74 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4202560 32673 0 0 0 1067 7 0 0 25 0 5 0 599733569 770281472 133883 33554432000 4194304 4338100 140735006639168 18446744073709551615 218880261577 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=905/tid=915] ppid=903 vsize=752228 CPUtime=10.75 cores=0,2,4,6
/proc/905/task/915/stat : 915 (ttsth-5-4) R 903 905 32461 0 -1 4202560 32684 0 0 0 1067 8 0 0 25 0 5 0 599733569 770281472 133883 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 44.75
Current children cumulated vsize (KiB) 752228

[startup+25.5008 s]
/proc/loadavg: 6.96 7.74 7.89 10/186 915
/proc/meminfo: memFree=29601536/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=1075772 CPUtime=95.07 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) S 903 905 32461 0 -1 4202496 228229 0 0 0 9449 58 0 0 25 0 5 0 599733385 1101590528 223503 33554432000 4194304 4338100 140735006639168 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 268943 223503 176 36 0 266848 0
[pid=905/tid=912] ppid=903 vsize=1075772 CPUtime=23.13 cores=0,2,4,6
/proc/905/task/912/stat : 912 (ttsth-5-4) R 903 905 32461 0 -1 4202560 63061 0 0 0 2297 16 0 0 25 0 5 0 599733569 1101590528 223503 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=905/tid=913] ppid=903 vsize=1075772 CPUtime=23.28 cores=0,2,4,6
/proc/905/task/913/stat : 913 (ttsth-5-4) R 903 905 32461 0 -1 4202560 48823 0 0 0 2316 12 0 0 21 0 5 0 599733569 1101590528 223503 33554432000 4194304 4338100 140735006639168 18446744073709551615 218880261577 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=905/tid=914] ppid=903 vsize=1075772 CPUtime=23.43 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4202560 54283 0 0 0 2330 13 0 0 18 0 5 0 599733569 1101590528 223503 33554432000 4194304 4338100 140735006639168 18446744073709551615 4285802 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=905/tid=915] ppid=903 vsize=1075772 CPUtime=23.36 cores=0,2,4,6
/proc/905/task/915/stat : 915 (ttsth-5-4) R 903 905 32461 0 -1 4202560 53818 0 0 0 2322 14 0 0 25 0 5 0 599733569 1101590528 223503 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 95.07
Current children cumulated vsize (KiB) 1075772

[startup+51.1004 s]
/proc/loadavg: 7.32 7.76 7.89 10/186 916
/proc/meminfo: memFree=29005800/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=1419496 CPUtime=195.83 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) S 903 905 32461 0 -1 4202496 327221 0 0 0 19490 93 0 0 25 0 5 0 599733385 1453563904 322495 33554432000 4194304 4338100 140735006639168 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 354874 322495 176 36 0 352779 0
[pid=905/tid=912] ppid=903 vsize=1419496 CPUtime=48.16 cores=0,2,4,6
/proc/905/task/912/stat : 912 (ttsth-5-4) R 903 905 32461 0 -1 4202560 99324 0 0 0 4787 29 0 0 19 0 5 0 599733569 1453563904 322495 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=905/tid=913] ppid=903 vsize=1419496 CPUtime=48.49 cores=0,2,4,6
/proc/905/task/913/stat : 913 (ttsth-5-4) R 903 905 32461 0 -1 4202560 77178 0 0 0 4828 21 0 0 25 0 5 0 599733569 1453563904 322495 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=905/tid=914] ppid=903 vsize=1419496 CPUtime=48.71 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4202560 71123 0 0 0 4852 19 0 0 25 0 5 0 599733569 1453563904 322495 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=905/tid=915] ppid=903 vsize=1419496 CPUtime=48.61 cores=0,2,4,6
/proc/905/task/915/stat : 915 (ttsth-5-4) R 903 905 32461 0 -1 4202560 71352 0 0 0 4839 22 0 0 25 0 5 0 599733569 1453563904 322495 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254239 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 195.83
Current children cumulated vsize (KiB) 1419496

[startup+102.306 s]
/proc/loadavg: 7.76 7.81 7.90 7/186 919
/proc/meminfo: memFree=28092132/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=2140080 CPUtime=397.47 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) S 903 905 32461 0 -1 4202496 483736 0 0 0 39597 150 0 0 25 0 5 0 599733385 2191441920 479010 33554432000 4194304 4338100 140735006639168 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 535020 479010 176 36 0 532925 0
[pid=905/tid=912] ppid=903 vsize=2140080 CPUtime=98.17 cores=0,2,4,6
/proc/905/task/912/stat : 912 (ttsth-5-4) R 903 905 32461 0 -1 4202560 159374 0 0 0 9771 46 0 0 23 0 5 0 599733569 2191441920 479010 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=905/tid=913] ppid=903 vsize=2140080 CPUtime=99.03 cores=0,2,4,6
/proc/905/task/913/stat : 913 (ttsth-5-4) R 903 905 32461 0 -1 4202560 115887 0 0 0 9868 35 0 0 25 0 5 0 599733569 2191441920 479010 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=905/tid=914] ppid=903 vsize=2140080 CPUtime=99.24 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4202560 105801 0 0 0 9893 31 0 0 25 0 5 0 599733569 2191441920 479010 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=905/tid=915] ppid=903 vsize=2140080 CPUtime=99.15 cores=0,2,4,6
/proc/905/task/915/stat : 915 (ttsth-5-4) R 903 905 32461 0 -1 4202560 94430 0 0 0 9880 35 0 0 25 0 5 0 599733569 2191441920 479010 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254433 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 397.47
Current children cumulated vsize (KiB) 2140080

[startup+162.301 s]
/proc/loadavg: 8.10 7.91 7.93 9/186 922
/proc/meminfo: memFree=27266880/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=2577460 CPUtime=632.51 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) S 903 905 32461 0 -1 4202496 620825 0 0 0 63044 207 0 0 25 0 5 0 599733385 2639319040 616099 33554432000 4194304 4338100 140735006639168 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 644365 616099 176 36 0 642270 0
[pid=905/tid=912] ppid=903 vsize=2577460 CPUtime=156.34 cores=0,2,4,6
/proc/905/task/912/stat : 912 (ttsth-5-4) R 903 905 32461 0 -1 4202560 231016 0 0 0 15564 70 0 0 25 0 5 0 599733569 2639319040 616099 33554432000 4194304 4338100 140735006639168 18446744073709551615 4235988 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=905/tid=913] ppid=903 vsize=2577460 CPUtime=157.87 cores=0,2,4,6
/proc/905/task/913/stat : 913 (ttsth-5-4) R 903 905 32461 0 -1 4202560 142126 0 0 0 15739 48 0 0 25 0 5 0 599733569 2639319040 616099 33554432000 4194304 4338100 140735006639168 18446744073709551615 4235651 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=905/tid=914] ppid=903 vsize=2577460 CPUtime=158.25 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4202560 130598 0 0 0 15784 41 0 0 19 0 5 0 599733569 2639319040 616099 33554432000 4194304 4338100 140735006639168 18446744073709551615 4235746 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=905/tid=915] ppid=903 vsize=2577460 CPUtime=158.17 cores=0,2,4,6
/proc/905/task/915/stat : 915 (ttsth-5-4) R 903 905 32461 0 -1 4202560 108841 0 0 0 15773 44 0 0 25 0 5 0 599733569 2639319040 616099 33554432000 4194304 4338100 140735006639168 18446744073709551615 4235651 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 632.51
Current children cumulated vsize (KiB) 2577460

[startup+222.3 s]
/proc/loadavg: 8.07 7.94 7.94 10/186 923
/proc/meminfo: memFree=26597000/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=3066276 CPUtime=867.79 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) S 903 905 32461 0 -1 4202496 716691 0 0 0 86520 259 0 0 25 0 5 0 599733385 3139866624 711965 33554432000 4194304 4338100 140735006639168 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 766569 711966 176 36 0 764474 0
[pid=905/tid=912] ppid=903 vsize=3066276 CPUtime=214.34 cores=0,2,4,6
/proc/905/task/912/stat : 912 (ttsth-5-4) R 903 905 32461 0 -1 4202560 278436 0 0 0 21342 92 0 0 18 0 5 0 599733569 3139866624 711966 33554432000 4194304 4338100 140735006639168 18446744073709551615 218879960506 0 0 4096 0 0 0 0 -1 6 0 0 0

################
# More data... #
################

/proc/905/task/912/stat : 912 (ttsth-5-4) R 903 905 32461 0 -1 4202560 664331 0 0 0 73350 262 0 0 21 0 5 0 599733569 5786636288 1370204 33554432000 4194304 4338100 140735006639168 18446744073709551615 4235746 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=905/tid=913] ppid=903 vsize=5651012 CPUtime=747.4 cores=0,2,4,6
/proc/905/task/913/stat : 913 (ttsth-5-4) R 903 905 32461 0 -1 4202560 260130 0 0 0 74608 132 0 0 25 0 5 0 599733569 5786636288 1370204 33554432000 4194304 4338100 140735006639168 18446744073709551615 4235674 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=905/tid=914] ppid=903 vsize=5651012 CPUtime=747.74 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4202560 227465 0 0 0 74646 128 0 0 25 0 5 0 599733569 5786636288 1370204 33554432000 4194304 4338100 140735006639168 18446744073709551615 4235651 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=905/tid=915] ppid=903 vsize=5651012 CPUtime=747.4 cores=0,2,4,6
/proc/905/task/915/stat : 915 (ttsth-5-4) R 903 905 32461 0 -1 4202560 214760 0 0 0 74592 148 0 0 25 0 5 0 599733569 5786636288 1370204 33554432000 4194304 4338100 140735006639168 18446744073709551615 4235847 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2980.53
Current children cumulated vsize (KiB) 5651012

[startup+822.301 s]
/proc/loadavg: 8.07 8.00 7.96 9/186 939
/proc/meminfo: memFree=22164136/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=6036492 CPUtime=3215.24 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) S 903 905 32461 0 -1 4202496 1449285 0 0 0 320805 719 0 0 25 0 5 0 599733385 6181367808 1444559 33554432000 4194304 4338100 140735006639168 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 1509123 1444559 176 36 0 1507028 0
[pid=905/tid=912] ppid=903 vsize=6036492 CPUtime=794.05 cores=0,2,4,6
/proc/905/task/912/stat : 912 (ttsth-5-4) R 903 905 32461 0 -1 4202560 711743 0 0 0 79121 284 0 0 25 0 5 0 599733569 6181367808 1444559 33554432000 4194304 4338100 140735006639168 18446744073709551615 218880261859 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=905/tid=913] ppid=903 vsize=6036492 CPUtime=806.3 cores=0,2,4,6
/proc/905/task/913/stat : 913 (ttsth-5-4) R 903 905 32461 0 -1 4202560 268421 0 0 0 80493 137 0 0 19 0 5 0 599733569 6181367808 1444559 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=905/tid=914] ppid=903 vsize=6036492 CPUtime=806.68 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4202560 238512 0 0 0 80531 137 0 0 18 0 5 0 599733569 6181367808 1444559 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=905/tid=915] ppid=903 vsize=6036492 CPUtime=806.33 cores=0,2,4,6
/proc/905/task/915/stat : 915 (ttsth-5-4) R 903 905 32461 0 -1 4202560 222365 0 0 0 80476 157 0 0 25 0 5 0 599733569 6181367808 1444559 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3215.24
Current children cumulated vsize (KiB) 6036492

[startup+882.3 s]
/proc/loadavg: 7.89 7.97 7.95 10/186 942
/proc/meminfo: memFree=21751048/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=6300764 CPUtime=3448.81 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) S 903 905 32461 0 -1 4202496 1523984 0 0 0 344114 767 0 0 25 0 5 0 599733385 6451982336 1519258 33554432000 4194304 4338100 140735006639168 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 1575191 1519258 176 36 0 1573096 0
[pid=905/tid=912] ppid=903 vsize=6300764 CPUtime=851.76 cores=0,2,4,6
/proc/905/task/912/stat : 912 (ttsth-5-4) R 903 905 32461 0 -1 4202560 757148 0 0 0 84871 305 0 0 17 0 5 0 599733569 6451982336 1519258 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254239 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=905/tid=913] ppid=903 vsize=6300764 CPUtime=865.1 cores=0,2,4,6
/proc/905/task/913/stat : 913 (ttsth-5-4) R 903 905 32461 0 -1 4202560 279610 0 0 0 86366 144 0 0 25 0 5 0 599733569 6451982336 1519258 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=905/tid=914] ppid=903 vsize=6300764 CPUtime=865.24 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4202560 246960 0 0 0 86377 147 0 0 25 0 5 0 599733569 6451982336 1519258 33554432000 4194304 4338100 140735006639168 18446744073709551615 218880261561 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=905/tid=915] ppid=903 vsize=6300764 CPUtime=864.85 cores=0,2,4,6
/proc/905/task/915/stat : 915 (ttsth-5-4) R 903 905 32461 0 -1 4202560 232022 0 0 0 86317 168 0 0 25 0 5 0 599733569 6451982336 1519258 33554432000 4194304 4338100 140735006639168 18446744073709551615 4251298 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3448.81
Current children cumulated vsize (KiB) 6300764

[startup+942.301 s]
/proc/loadavg: 8.00 7.99 7.96 9/186 944
/proc/meminfo: memFree=21351852/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=6731776 CPUtime=3683.5 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) S 903 905 32461 0 -1 4202496 1593853 0 0 0 367540 810 0 0 25 0 5 0 599733385 6893338624 1589127 33554432000 4194304 4338100 140735006639168 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 1682944 1589127 176 36 0 1680849 0
[pid=905/tid=912] ppid=903 vsize=6731776 CPUtime=909.7 cores=0,2,4,6
/proc/905/task/912/stat : 912 (ttsth-5-4) R 903 905 32461 0 -1 4202560 795595 0 0 0 90646 324 0 0 25 0 5 0 599733569 6893338624 1589127 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=905/tid=913] ppid=903 vsize=6731776 CPUtime=924.24 cores=0,2,4,6
/proc/905/task/913/stat : 913 (ttsth-5-4) R 903 905 32461 0 -1 4202560 288858 0 0 0 92275 149 0 0 25 0 5 0 599733569 6893338624 1589127 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=905/tid=914] ppid=903 vsize=6731776 CPUtime=924.09 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4202560 258143 0 0 0 92254 155 0 0 25 0 5 0 599733569 6893338624 1589127 33554432000 4194304 4338100 140735006639168 18446744073709551615 4256139 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=905/tid=915] ppid=903 vsize=6731776 CPUtime=923.58 cores=0,2,4,6
/proc/905/task/915/stat : 915 (ttsth-5-4) R 903 905 32461 0 -1 4202560 243013 0 0 0 92180 178 0 0 25 0 5 0 599733569 6893338624 1589127 33554432000 4194304 4338100 140735006639168 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3683.5
Current children cumulated vsize (KiB) 6731776

[startup+1002.3 s]
/proc/loadavg: 7.63 7.89 7.93 9/186 945
/proc/meminfo: memFree=21031996/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=6888048 CPUtime=3917.53 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) S 903 905 32461 0 -1 4202496 1661759 0 0 0 390899 854 0 0 25 0 5 0 599733385 7053361152 1657033 33554432000 4194304 4338100 140735006639168 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 1722012 1657033 176 36 0 1719917 0
[pid=905/tid=912] ppid=903 vsize=6888048 CPUtime=967.44 cores=0,2,4,6
/proc/905/task/912/stat : 912 (ttsth-5-4) R 903 905 32461 0 -1 4202560 834441 0 0 0 96400 344 0 0 25 0 5 0 599733569 7053361152 1657033 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=905/tid=913] ppid=903 vsize=6888048 CPUtime=983.03 cores=0,2,4,6
/proc/905/task/913/stat : 913 (ttsth-5-4) R 903 905 32461 0 -1 4202560 298407 0 0 0 98144 159 0 0 25 0 5 0 599733569 7053361152 1657033 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=905/tid=914] ppid=903 vsize=6888048 CPUtime=982.91 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4202560 266731 0 0 0 98128 163 0 0 25 0 5 0 599733569 7053361152 1657033 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=905/tid=915] ppid=903 vsize=6888048 CPUtime=982.27 cores=0,2,4,6
/proc/905/task/915/stat : 915 (ttsth-5-4) R 903 905 32461 0 -1 4202560 253936 0 0 0 98042 185 0 0 25 0 5 0 599733569 7053361152 1657033 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3917.53
Current children cumulated vsize (KiB) 6888048

[startup+1062.3 s]
/proc/loadavg: 7.91 7.92 7.93 10/186 947
/proc/meminfo: memFree=20635788/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=7244676 CPUtime=4152.35 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) S 903 905 32461 0 -1 4202496 1725976 0 0 0 414340 895 0 0 25 0 5 0 599733385 7418548224 1721250 33554432000 4194304 4338100 140735006639168 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 1811169 1721250 176 36 0 1809074 0
[pid=905/tid=912] ppid=903 vsize=7244676 CPUtime=1025.36 cores=0,2,4,6
/proc/905/task/912/stat : 912 (ttsth-5-4) R 903 905 32461 0 -1 4202560 874452 0 0 0 102176 360 0 0 25 0 5 0 599733569 7418548224 1721250 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254321 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=905/tid=913] ppid=903 vsize=7244676 CPUtime=1041.88 cores=0,2,4,6
/proc/905/task/913/stat : 913 (ttsth-5-4) R 903 905 32461 0 -1 4202560 305850 0 0 0 104021 167 0 0 25 0 5 0 599733569 7418548224 1721250 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=905/tid=914] ppid=903 vsize=7244676 CPUtime=1041.92 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4202560 274152 0 0 0 104019 173 0 0 18 0 5 0 599733569 7418548224 1721250 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=905/tid=915] ppid=903 vsize=7244676 CPUtime=1041.31 cores=0,2,4,6
/proc/905/task/915/stat : 915 (ttsth-5-4) R 903 905 32461 0 -1 4202560 263278 0 0 0 103939 192 0 0 25 0 5 0 599733569 7418548224 1721250 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4152.35
Current children cumulated vsize (KiB) 7244676

[startup+1122.3 s]
/proc/loadavg: 7.84 7.89 7.91 10/186 948
/proc/meminfo: memFree=20171620/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=7412760 CPUtime=4386.65 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) S 903 905 32461 0 -1 4202496 1790963 0 0 0 437727 938 0 0 25 0 5 0 599733385 7590666240 1786237 33554432000 4194304 4338100 140735006639168 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 1853190 1786237 176 36 0 1851095 0
[pid=905/tid=912] ppid=903 vsize=7412760 CPUtime=1083.02 cores=0,2,4,6
/proc/905/task/912/stat : 912 (ttsth-5-4) R 903 905 32461 0 -1 4202560 916340 0 0 0 107921 381 0 0 25 0 5 0 599733569 7590666240 1786237 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=905/tid=913] ppid=903 vsize=7412760 CPUtime=1100.63 cores=0,2,4,6
/proc/905/task/913/stat : 913 (ttsth-5-4) R 903 905 32461 0 -1 4202560 314079 0 0 0 109892 171 0 0 21 0 5 0 599733569 7590666240 1786237 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=905/tid=914] ppid=903 vsize=7412760 CPUtime=1100.67 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4202560 281294 0 0 0 109885 182 0 0 25 0 5 0 599733569 7590666240 1786237 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=905/tid=915] ppid=903 vsize=7412760 CPUtime=1100.47 cores=0,2,4,6
/proc/905/task/915/stat : 915 (ttsth-5-4) R 903 905 32461 0 -1 4202560 271006 0 0 0 109846 201 0 0 25 0 5 0 599733569 7590666240 1786237 33554432000 4194304 4338100 140735006639168 18446744073709551615 218879960518 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4386.65
Current children cumulated vsize (KiB) 7412760

[startup+1182.3 s]
/proc/loadavg: 7.98 7.92 7.92 9/186 950
/proc/meminfo: memFree=19905092/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=7827732 CPUtime=4622.01 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) S 903 905 32461 0 -1 4202496 1847969 0 0 0 461222 979 0 0 25 0 5 0 599733385 8015597568 1843243 33554432000 4194304 4338100 140735006639168 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 1956933 1843243 176 36 0 1954838 0
[pid=905/tid=912] ppid=903 vsize=7827732 CPUtime=1141.22 cores=0,2,4,6
/proc/905/task/912/stat : 912 (ttsth-5-4) R 903 905 32461 0 -1 4202560 945833 0 0 0 113725 397 0 0 25 0 5 0 599733569 8015597568 1843243 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=905/tid=913] ppid=903 vsize=7827732 CPUtime=1159.73 cores=0,2,4,6
/proc/905/task/913/stat : 913 (ttsth-5-4) R 903 905 32461 0 -1 4202560 323966 0 0 0 115795 178 0 0 25 0 5 0 599733569 8015597568 1843243 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=905/tid=914] ppid=903 vsize=7827732 CPUtime=1159.64 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4202560 288698 0 0 0 115773 191 0 0 25 0 5 0 599733569 8015597568 1843243 33554432000 4194304 4338100 140735006639168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=905/tid=915] ppid=903 vsize=7827732 CPUtime=1159.56 cores=0,2,4,6
/proc/905/task/915/stat : 915 (ttsth-5-4) R 903 905 32461 0 -1 4202560 281228 0 0 0 115746 210 0 0 25 0 5 0 599733569 8015597568 1843243 33554432000 4194304 4338100 140735006639168 18446744073709551615 4240519 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4622.01
Current children cumulated vsize (KiB) 7827732



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1227.8 s]
/proc/loadavg: 6.54 7.59 7.81 2/173 969
/proc/meminfo: memFree=23712108/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=0 CPUtime=4800.02 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) Z 903 905 32461 0 -1 4203548 1887098 0 0 0 478956 1046 0 0 18 0 2 0 599733385 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 0 0 0 0 0 0 0
[pid=905/tid=914] ppid=903 vsize=0 CPUtime=1204.56 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4203588 297306 0 0 0 120254 202 0 0 25 0 2 0 599733569 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.02
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1228 s]
/proc/loadavg: 6.54 7.59 7.81 2/173 969
/proc/meminfo: memFree=23712108/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=0 CPUtime=4800.22 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) Z 903 905 32461 0 -1 4203548 1887098 0 0 0 478956 1066 0 0 18 0 2 0 599733385 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 0 0 0 0 0 0 0
[pid=905/tid=914] ppid=903 vsize=0 CPUtime=1204.75 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4203588 297306 0 0 0 120254 221 0 0 25 0 2 0 599733569 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.22
Current children cumulated vsize (KiB) 0

[startup+1228.1 s]
/proc/loadavg: 6.54 7.59 7.81 2/173 969
/proc/meminfo: memFree=23712108/32950928 swapFree=67111524/67111528
[pid=905] ppid=903 vsize=0 CPUtime=4800.32 cores=0,2,4,6
/proc/905/stat : 905 (ttsth-5-4) Z 903 905 32461 0 -1 4203548 1887098 0 0 0 478956 1076 0 0 18 0 2 0 599733385 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/905/statm: 0 0 0 0 0 0 0
[pid=905/tid=914] ppid=903 vsize=0 CPUtime=1204.85 cores=0,2,4,6
/proc/905/task/914/stat : 914 (ttsth-5-4) R 903 905 32461 0 -1 4203588 297306 0 0 0 120254 231 0 0 25 0 2 0 599733569 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.32
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1228.12
CPU time (s): 4800.35
CPU user time (s): 4789.57
CPU system time (s): 10.7854
CPU usage (%): 390.87
Max. virtual memory (cumulated for all children) (KiB): 7961692

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4789.57
system time used= 10.7854
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1887098
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 163448
involuntary context switches= 21790

runsolver used 2.02969 second user time and 4.33534 second system time

The end

Launcher Data 

Begin job on node112 at 2011-04-29 18:37:07
IDJOB=3262882
IDBENCH=83899
IDSOLVER=1654
FILE ID=node112/3262882-1304095027
RUNJOBID= node112-1304095027-885
PBS_JOBID= 13170569
Free space on /tmp= 72564 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT03/handmade/anton/UNSAT/SGI_30_70_19_80_8-log.shuffled-as.sat03-143.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3262882-1304095027/watcher-3262882-1304095027 -o /tmp/evaluation-result-3262882-1304095027/solver-3262882-1304095027 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262882-1304095027.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 3aff0c2163848e3eed6bc22fc2c30cdf
RANDOM SEED=1308197822

node112.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5261.27
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30878464 kB
Buffers:        439252 kB
Cached:        1162908 kB
SwapCached:          4 kB
Active:         135920 kB
Inactive:      1536972 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30878464 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:            2480 kB
Writeback:           0 kB
AnonPages:       70336 kB
Mapped:          14884 kB
Slab:           334580 kB
PageTables:       5144 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   213668 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72564 MiB
End job on node112 at 2011-04-29 18:57:37