Trace number 3259670

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.26 1203.26

General information on the benchmark

NameCRAFTED/rbsat/crafted/
forced/rbsat-v2640c305320gyes10.cnf
MD5SUM98bac4f2d533fe7abb12552567f9e1aa
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2640
Number of clauses305320
Sum of the clauses size613120
Maximum clause length33
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2305240
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 580

Solver Data 

0.19/0.21	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259670-1304056189.cnf 
0.19/0.21	c Using SATTIMEOUT of 4800
0.19/0.21	c Using SATRAM of 15500
0.19/0.21	c Size of subset node = 32
0.19/0.21	c Variables = 2640
0.19/0.21	c Clauses = 305320
0.19/0.21	c Literals = 613120
0.19/0.21	c Reduced size:
0.19/0.21	c Variables = 2640
0.19/0.21	c Clauses = 305320
0.19/0.21	c Literals = 613120
0.19/0.21	c a2
0.28/0.38	c a1
0.79/0.81	c Independent sub-problems = 1
0.79/0.81	c Sub-problem with 2640 variables and 305320 clauses
0.79/0.81	c using permutation method 4
0.79/0.81	c count = 1
0.79/0.81	c Ordering finished 0.760 seconds
0.79/0.81	c mapped 0.800 seconds
0.79/0.81	c permuted 0.800 seconds
0.79/0.81	c about to sort
0.79/0.83	c sorted
1.59/1.62	c clauses now 305320
1.59/1.62	c init finished 0.900 seconds
1.59/1.62	c tree built 1.620 seconds
1.59/1.62	c 
1.59/1.62	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
1.59/1.62	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
1.59/1.62	c ................................ | ..........                    |                                  1.6         0.0             
1.59/1.63	c ................................ | ..........                  1 |                                  1.6         0.6             
1.59/1.63	c ................................ | ..........                  2 |                                  1.6         1.2             
1.59/1.63	c ................................ | ..........                  3 |                                  1.6         1.9             
1.59/1.63	c ................................ | ..........                  4 |                                  1.6         2.5             
1.59/1.63	c ................................ | ..........                  5 |                                  1.6         3.1             
1.59/1.63	c ................................ | ..........               1000 |              102                 1.6       609.8         4393
95.77/25.26	c ................................ | ..........            2758000 | 41749        221791             96.0     28723.2     19156556
192.02/49.30	c ................................ | ..........            5052000 | 77055        391397            192.0     26309.8     33256175
287.90/73.33	c ................................ | ..........            7028000 | 108408       545870            288.0     24401.1     43115729
383.74/97.38	c ................................ | ..........            8622000 | 141980       677370            384.1     22449.0     48912980
479.99/121.40	c ................................ | *.........           10208000 | 171174       804412            480.0     21265.8     56311644
575.85/145.44	c ................................ | *.........           12058000 | 199569       926194            576.0     20932.6     65497574
671.73/169.48	c ................................ | *.........           13665000 | 226429       1038307           672.0     20333.6     68761524
767.97/193.52	c ................................ | *.........           15135000 | 252585       1138285           768.0     19705.7     71964028
863.81/217.56	c ................................ | *.........           16748000 | 275777       1241719           864.1     19382.9     76590670
959.67/241.58	c ................................ | **........           18407000 | 297575       1340557           960.0     19174.0     84402829
1055.94/265.61	c ................................ | **........           19784000 | 319830       1428933          1056.0     18734.8     89977034
1151.79/289.65	c ................................ | **........           21130000 | 340882       1520691          1152.0     18341.9     94696978
1248.04/313.70	c ................................ | **........           22450000 | 361581       1603762          1248.0     17988.1     96190369
1343.93/337.73	c ................................ | **........           23687000 | 383166       1685867          1344.0     17624.0    100241035
1439.77/361.76	c ................................ | ***.......           25080000 | 406381       1775489          1440.0     17416.3    102877633
1536.04/385.80	c ................................ | ***.......           26451000 | 427946       1865051          1536.0     17220.3    108054341
1631.91/409.83	c ................................ | ***.......           27694000 | 448390       1950804          1632.0     16968.8    114373763
1727.78/433.86	c ................................ | ***.......           28889000 | 469274       2029428          1728.0     16717.7    118294528
1824.02/457.90	c ................................ | ***.......           30042000 | 490219       2103136          1824.0     16469.9    123440223
1919.89/481.93	c ................................ | ****......           31077000 | 510021       2177269          1920.0     16185.7    123707196
2015.78/505.97	c ................................ | ****......           32177000 | 530582       2246059          2016.0     15960.5    125720963
2112.01/530.01	c ................................ | ****......           33370000 | 550021       2319061          2112.1     15799.7    129498889
2207.84/554.05	c ................................ | ****......           34766000 | 570649       2394132          2208.1     15745.1    133849126
2303.67/578.08	c ................................ | ****......           36110000 | 590474       2466200          2304.0     15672.7    138366222
2399.94/602.13	c ................................ | *****.....           37460000 | 609949       2538395          2400.1     15607.9    139474950
2495.80/626.15	c ................................ | *****.....           38634000 | 627335       2605190          2496.0     15478.2    142107311
2591.68/650.18	c ................................ | *****.....           39853000 | 645948       2673390          2592.0     15375.3    146049647
2687.91/674.24	c ................................ | *****.....           41093000 | 666853       2743927          2688.1     15287.2    149162314
2783.76/698.26	c ................................ | *****.....           42294000 | 686194       2820600          2784.0     15191.6    154051362
2880.03/722.31	c ................................ | ******....           43547000 | 705215       2891460          2880.1     15120.0    158035489
2975.88/746.35	c ................................ | ******....           44692000 | 725194       2962077          2976.1     15016.9    162950505
3071.75/770.37	c ................................ | ******....           45923000 | 744724       3042645          3072.1     14948.6    168667020
3168.02/794.40	c ................................ | ******....           47052000 | 765263       3113150          3168.0     14852.1    172856544
3263.89/818.46	c ................................ | ******....           48084000 | 782152       3182236          3264.1     14731.2    175176736
3359.70/842.48	c ................................ | *******...           49108000 | 799695       3253946          3360.1     14615.3    175176736
3455.97/866.52	c ................................ | *******...           50107000 | 818451       3323006          3456.1     14498.3    175176736
3551.83/890.56	c ................................ | *******...           51134000 | 838700       3389405          3552.1     14395.5    175176736
3648.03/914.60	c ................................ | *******...           52194000 | 856040       3456751          3648.1     14307.4    175176736
3743.88/938.64	c ................................ | *******...           53314000 | 875355       3525026          3744.1     14239.6    175176736
3839.75/962.67	c ................................ | ********..           54330000 | 894514       3597312          3840.0     14148.4    175176736
3935.64/986.70	c ................................ | ********..           55287000 | 911287       3664793          3936.0     14046.4    175176736
4031.89/1010.75	c ................................ | ********..           56339000 | 925524       3734834          4032.1     13972.7    175176736
4127.75/1034.78	c ................................ | ********..           57307000 | 941160       3797394          4128.1     13882.2    175258554
4224.02/1058.81	c ................................ | ********..           58252000 | 956723       3863225          4224.1     13790.5    176836361
4319.87/1082.83	c ................................ | *********.           59191000 | 972348       3931451          4320.0     13701.6    179587172
4415.70/1106.89	c ................................ | *********.           60209000 | 987724       3993232          4416.1     13634.1    180331114
4511.98/1130.91	c ................................ | *********.           61147000 | 1002578      4051601          4512.0     13552.0    182952691
4607.84/1154.96	c ................................ | *********.           62074000 | 1017508      4114251          4608.1     13470.7    185739859
4703.68/1178.99	c ................................ | *********.           63039000 | 1033529      4172621          4704.0     13401.0    186881716
4799.94/1203.01	c CPU limit (4800) reached
4799.94/1203.01	c Time = 4800.01 seconds
4799.94/1203.01	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3259670-1304056189/watcher-3259670-1304056189 -o /tmp/evaluation-result-3259670-1304056189/solver-3259670-1304056189 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259670-1304056189.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.05 8.09 8.04 2/175 30766
/proc/meminfo: memFree=20506188/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=9540 CPUtime=0 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) R 30764 30766 30360 0 -1 4202496 458 0 0 0 0 0 0 0 25 0 1 0 595849574 9768960 401 33554432000 4194304 4338100 140736497524768 18446744073709551615 231895151301 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30766/statm: 2385 402 147 36 0 290 0

[startup+0.0839029 s]
/proc/loadavg: 8.05 8.09 8.04 2/175 30766
/proc/meminfo: memFree=20506188/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=25508 CPUtime=0.07 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) R 30764 30766 30360 0 -1 4202496 4308 0 0 0 7 0 0 0 25 0 1 0 595849574 26120192 4251 33554432000 4194304 4338100 140736497524768 18446744073709551615 231895151301 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30766/statm: 6377 4254 148 36 0 4282 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 25508

[startup+0.100894 s]
/proc/loadavg: 8.05 8.09 8.04 2/175 30766
/proc/meminfo: memFree=20506188/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=28676 CPUtime=0.09 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) R 30764 30766 30360 0 -1 4202496 5149 0 0 0 9 0 0 0 25 0 1 0 595849574 29364224 5092 33554432000 4194304 4338100 140736497524768 18446744073709551615 231895151301 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30766/statm: 7169 5094 148 36 0 5074 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 28676

[startup+0.300896 s]
/proc/loadavg: 8.05 8.09 8.04 2/175 30766
/proc/meminfo: memFree=20506188/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=67596 CPUtime=0.28 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) R 30764 30766 30360 0 -1 4202496 14945 0 0 0 27 1 0 0 25 0 1 0 595849574 69218304 14887 33554432000 4194304 4338100 140736497524768 18446744073709551615 4206993 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30766/statm: 16899 14887 153 36 0 14804 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 67596

[startup+0.700799 s]
/proc/loadavg: 8.05 8.09 8.04 2/175 30766
/proc/meminfo: memFree=20506188/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=77056 CPUtime=0.69 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) R 30764 30766 30360 0 -1 4202496 18524 0 0 0 67 2 0 0 25 0 1 0 595849574 78905344 17272 33554432000 4194304 4338100 140736497524768 18446744073709551615 4295924 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30766/statm: 19264 17272 156 36 0 17169 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 77056

[startup+1.50065 s]
/proc/loadavg: 8.05 8.09 8.04 3/178 30794
/proc/meminfo: memFree=20354176/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=181052 CPUtime=1.49 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) R 30764 30766 30360 0 -1 4202496 45109 0 0 0 143 6 0 0 25 0 1 0 595849574 185397248 43260 33554432000 4194304 4338100 140736497524768 18446744073709551615 231895151301 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30766/statm: 45263 43264 164 36 0 43168 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 181052

[startup+3.10027 s]
/proc/loadavg: 7.57 7.99 8.01 9/186 30802
/proc/meminfo: memFree=20111816/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=434628 CPUtime=7.47 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) S 30764 30766 30360 0 -1 4202496 64997 0 0 0 724 23 0 0 25 0 5 0 595849574 445059072 62550 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 108657 62551 176 36 0 106562 0
[pid=30766/tid=30795] ppid=30764 vsize=434628 CPUtime=1.45 cores=1,3,5,7
/proc/30766/task/30795/stat : 30795 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 4358 0 0 0 144 1 0 0 20 0 5 0 595849737 445059072 62551 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30766/tid=30796] ppid=30764 vsize=434628 CPUtime=1.46 cores=1,3,5,7
/proc/30766/task/30796/stat : 30796 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 3014 0 0 0 138 8 0 0 20 0 5 0 595849737 445059072 62551 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30766/tid=30797] ppid=30764 vsize=434628 CPUtime=1.45 cores=1,3,5,7
/proc/30766/task/30797/stat : 30797 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 3952 0 0 0 145 0 0 0 20 0 5 0 595849737 445059072 62551 33554432000 4194304 4338100 140736497524768 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30766/tid=30798] ppid=30764 vsize=434628 CPUtime=1.45 cores=1,3,5,7
/proc/30766/task/30798/stat : 30798 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 3241 0 0 0 139 6 0 0 25 0 5 0 595849737 445059072 62551 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 7.47
Current children cumulated vsize (KiB) 434628

[startup+6.30068 s]
/proc/loadavg: 7.57 7.99 8.01 9/186 30802
/proc/meminfo: memFree=19934268/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=535152 CPUtime=20.26 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) S 30764 30766 30360 0 -1 4202496 90359 0 0 0 1974 52 0 0 25 0 5 0 595849574 547995648 87912 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 133788 87912 176 36 0 131693 0
[pid=30766/tid=30795] ppid=30764 vsize=535152 CPUtime=4.64 cores=1,3,5,7
/proc/30766/task/30795/stat : 30795 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 12090 0 0 0 461 3 0 0 25 0 5 0 595849737 547995648 87912 33554432000 4194304 4338100 140736497524768 18446744073709551615 4256007 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30766/tid=30796] ppid=30764 vsize=535152 CPUtime=4.66 cores=1,3,5,7
/proc/30766/task/30796/stat : 30796 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 7996 0 0 0 446 20 0 0 25 0 5 0 595849737 547995648 87912 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30766/tid=30797] ppid=30764 vsize=535152 CPUtime=4.66 cores=1,3,5,7
/proc/30766/task/30797/stat : 30797 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 11361 0 0 0 464 2 0 0 25 0 5 0 595849737 547995648 87913 33554432000 4194304 4338100 140736497524768 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30766/tid=30798] ppid=30764 vsize=535152 CPUtime=4.66 cores=1,3,5,7
/proc/30766/task/30798/stat : 30798 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 8480 0 0 0 447 19 0 0 25 0 5 0 595849737 547995648 87913 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 20.26
Current children cumulated vsize (KiB) 535152

[startup+12.7005 s]
/proc/loadavg: 7.79 8.02 8.02 10/186 30802
/proc/meminfo: memFree=19653836/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=704404 CPUtime=45.83 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) S 30764 30766 30360 0 -1 4202496 131853 0 0 0 4476 107 0 0 25 0 5 0 595849574 721309696 129406 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 176101 129407 176 36 0 174006 0
[pid=30766/tid=30795] ppid=30764 vsize=704404 CPUtime=11 cores=1,3,5,7
/proc/30766/task/30795/stat : 30795 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 23856 0 0 0 1094 6 0 0 25 0 5 0 595849737 721309696 129407 33554432000 4194304 4338100 140736497524768 18446744073709551615 4206915 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30766/tid=30796] ppid=30764 vsize=704404 CPUtime=11.05 cores=1,3,5,7
/proc/30766/task/30796/stat : 30796 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 16508 0 0 0 1060 45 0 0 25 0 5 0 595849737 721309696 129407 33554432000 4194304 4338100 140736497524768 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30766/tid=30797] ppid=30764 vsize=704404 CPUtime=11.06 cores=1,3,5,7
/proc/30766/task/30797/stat : 30797 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 23845 0 0 0 1102 4 0 0 25 0 5 0 595849737 721309696 129408 33554432000 4194304 4338100 140736497524768 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30766/tid=30798] ppid=30764 vsize=704404 CPUtime=11.06 cores=1,3,5,7
/proc/30766/task/30798/stat : 30798 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 17213 0 0 0 1063 43 0 0 25 0 5 0 595849737 721309696 129408 33554432000 4194304 4338100 140736497524768 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 45.83
Current children cumulated vsize (KiB) 704404

[startup+25.5011 s]
/proc/loadavg: 7.89 8.04 8.02 9/186 30803
/proc/meminfo: memFree=19144444/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=986508 CPUtime=96.95 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) S 30764 30766 30360 0 -1 4202496 203728 0 0 0 9475 220 0 0 25 0 5 0 595849574 1010184192 201281 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 246627 201281 176 36 0 244532 0
[pid=30766/tid=30795] ppid=30764 vsize=986508 CPUtime=23.73 cores=1,3,5,7
/proc/30766/task/30795/stat : 30795 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 42397 0 0 0 2362 11 0 0 25 0 5 0 595849737 1010184192 201281 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30766/tid=30796] ppid=30764 vsize=986508 CPUtime=23.86 cores=1,3,5,7
/proc/30766/task/30796/stat : 30796 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 30830 0 0 0 2289 97 0 0 25 0 5 0 595849737 1010184192 201281 33554432000 4194304 4338100 140736497524768 18446744073709551615 4240530 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30766/tid=30797] ppid=30764 vsize=986508 CPUtime=23.86 cores=1,3,5,7
/proc/30766/task/30797/stat : 30797 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 47853 0 0 0 2378 8 0 0 25 0 5 0 595849737 1010184192 201281 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30766/tid=30798] ppid=30764 vsize=986508 CPUtime=23.85 cores=1,3,5,7
/proc/30766/task/30798/stat : 30798 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 32215 0 0 0 2289 96 0 0 25 0 5 0 595849737 1010184192 201281 33554432000 4194304 4338100 140736497524768 18446744073709551615 4240530 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 96.95
Current children cumulated vsize (KiB) 986508

[startup+51.1003 s]
/proc/loadavg: 8.06 8.07 8.03 9/186 30803
/proc/meminfo: memFree=18386596/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=1459976 CPUtime=199.21 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) S 30764 30766 30360 0 -1 4202496 321416 0 0 0 19497 424 0 0 25 0 5 0 595849574 1495015424 318969 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 364994 318969 176 36 0 362899 0
[pid=30766/tid=30795] ppid=30764 vsize=1459976 CPUtime=49.18 cores=1,3,5,7
/proc/30766/task/30795/stat : 30795 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 77474 0 0 0 4896 22 0 0 25 0 5 0 595849737 1495015424 318970 33554432000 4194304 4338100 140736497524768 18446744073709551615 231894885428 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30766/tid=30796] ppid=30764 vsize=1459976 CPUtime=49.46 cores=1,3,5,7
/proc/30766/task/30796/stat : 30796 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 54233 0 0 0 4752 194 0 0 25 0 5 0 595849737 1495015424 318970 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254014 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30766/tid=30797] ppid=30764 vsize=1459976 CPUtime=49.46 cores=1,3,5,7
/proc/30766/task/30797/stat : 30797 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 85554 0 0 0 4932 14 0 0 25 0 5 0 595849737 1495015424 318970 33554432000 4194304 4338100 140736497524768 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30766/tid=30798] ppid=30764 vsize=1459976 CPUtime=49.45 cores=1,3,5,7
/proc/30766/task/30798/stat : 30798 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 53723 0 0 0 4760 185 0 0 25 0 5 0 595849737 1495015424 318970 33554432000 4194304 4338100 140736497524768 18446744073709551615 231894885857 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 199.21
Current children cumulated vsize (KiB) 1459976

[startup+102.307 s]
/proc/loadavg: 8.08 8.07 8.03 9/186 30805
/proc/meminfo: memFree=17334060/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=2011564 CPUtime=403.74 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) S 30764 30766 30360 0 -1 4202496 458995 0 0 0 39584 790 0 0 25 0 5 0 595849574 2059841536 453460 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 502891 453460 176 36 0 500796 0
[pid=30766/tid=30795] ppid=30764 vsize=2011564 CPUtime=100.11 cores=1,3,5,7
/proc/30766/task/30795/stat : 30795 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 124336 0 0 0 9974 37 0 0 25 0 5 0 595849737 2059841536 453461 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30766/tid=30796] ppid=30764 vsize=2011564 CPUtime=100.66 cores=1,3,5,7
/proc/30766/task/30796/stat : 30796 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 73928 0 0 0 9700 366 0 0 25 0 5 0 595849737 2059841536 453461 33554432000 4194304 4338100 140736497524768 18446744073709551615 231894885392 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30766/tid=30797] ppid=30764 vsize=2011564 CPUtime=100.66 cores=1,3,5,7
/proc/30766/task/30797/stat : 30797 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 134025 0 0 0 10042 24 0 0 25 0 5 0 595849737 2059841536 453461 33554432000 4194304 4338100 140736497524768 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30766/tid=30798] ppid=30764 vsize=2011564 CPUtime=100.66 cores=1,3,5,7
/proc/30766/task/30798/stat : 30798 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 76274 0 0 0 9711 355 0 0 25 0 5 0 595849737 2059841536 453461 33554432000 4194304 4338100 140736497524768 18446744073709551615 231894885606 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 403.74
Current children cumulated vsize (KiB) 2011564

[startup+162.3 s]
/proc/loadavg: 8.09 8.07 8.03 9/186 30806
/proc/meminfo: memFree=16374536/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=2572908 CPUtime=643.34 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) S 30764 30766 30360 0 -1 4202496 596804 0 0 0 63161 1173 0 0 25 0 5 0 595849574 2634657792 591269 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 643227 591269 176 36 0 641132 0
[pid=30766/tid=30795] ppid=30764 vsize=2572908 CPUtime=159.9 cores=1,3,5,7
/proc/30766/task/30795/stat : 30795 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 176665 0 0 0 15939 51 0 0 25 0 5 0 595849737 2634657792 591269 33554432000 4194304 4338100 140736497524768 18446744073709551615 231894885762 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30766/tid=30796] ppid=30764 vsize=2572908 CPUtime=160.64 cores=1,3,5,7
/proc/30766/task/30796/stat : 30796 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 92603 0 0 0 15519 545 0 0 25 0 5 0 595849737 2634657792 591269 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30766/tid=30797] ppid=30764 vsize=2572908 CPUtime=160.49 cores=1,3,5,7
/proc/30766/task/30797/stat : 30797 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 182177 0 0 0 16014 35 0 0 25 0 5 0 595849737 2634657792 591269 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30766/tid=30798] ppid=30764 vsize=2572908 CPUtime=160.64 cores=1,3,5,7
/proc/30766/task/30798/stat : 30798 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 94926 0 0 0 15531 533 0 0 25 0 5 0 595849737 2634657792 591269 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 643.34
Current children cumulated vsize (KiB) 2572908

[startup+222.3 s]
/proc/loadavg: 8.08 8.07 8.03 9/186 30808
/proc/meminfo: memFree=15686056/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=2920972 CPUtime=882.98 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) S 30764 30766 30360 0 -1 4202496 681787 0 0 0 86789 1509 0 0 25 0 5 0 595849574 2991075328 676252 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 730243 676252 176 36 0 728148 0
[pid=30766/tid=30795] ppid=30764 vsize=2920972 CPUtime=219.89 cores=1,3,5,7
/proc/30766/task/30795/stat : 30795 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 217471 0 0 0 21922 67 0 0 25 0 5 0 595849737 2991075328 676252 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0

################
# More data... #
################

/proc/30766/task/30798/stat : 30798 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 143580 0 0 0 68186 1863 0 0 25 0 5 0 595849737 5487128576 1283356 33554432000 4194304 4338100 140736497524768 18446744073709551615 4240496 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2800.14
Current children cumulated vsize (KiB) 5358524

[startup+762.306 s]
/proc/loadavg: 8.14 8.11 8.05 9/186 30831
/proc/meminfo: memFree=10585704/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=5737144 CPUtime=3039.81 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) S 30764 30766 30360 0 -1 4202496 1383257 0 0 0 299596 4385 0 0 25 0 5 0 595849574 5874835456 1377722 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 1434286 1377722 176 36 0 1432191 0
[pid=30766/tid=30795] ppid=30764 vsize=5737144 CPUtime=758.45 cores=1,3,5,7
/proc/30766/task/30795/stat : 30795 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 568973 0 0 0 75646 199 0 0 25 0 5 0 595849737 5874835456 1377723 33554432000 4194304 4338100 140736497524768 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30766/tid=30796] ppid=30764 vsize=5737144 CPUtime=759.69 cores=1,3,5,7
/proc/30766/task/30796/stat : 30796 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 165641 0 0 0 73906 2063 0 0 25 0 5 0 595849737 5874835456 1377723 33554432000 4194304 4338100 140736497524768 18446744073709551615 4257956 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30766/tid=30797] ppid=30764 vsize=5737144 CPUtime=759.84 cores=1,3,5,7
/proc/30766/task/30797/stat : 30797 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 444883 0 0 0 75890 94 0 0 25 0 5 0 595849737 5874835456 1377723 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254326 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30766/tid=30798] ppid=30764 vsize=5737144 CPUtime=760.16 cores=1,3,5,7
/proc/30766/task/30798/stat : 30798 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 153328 0 0 0 73996 2020 0 0 25 0 5 0 595849737 5874835456 1377723 33554432000 4194304 4338100 140736497524768 18446744073709551615 231894885606 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3039.81
Current children cumulated vsize (KiB) 5737144

[startup+822.301 s]
/proc/loadavg: 8.12 8.10 8.05 9/186 30833
/proc/meminfo: memFree=9690160/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=6204696 CPUtime=3279.41 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) S 30764 30766 30360 0 -1 4202496 1492370 0 0 0 323258 4683 0 0 25 0 5 0 595849574 6353608704 1486835 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 1551174 1486835 176 36 0 1549079 0
[pid=30766/tid=30795] ppid=30764 vsize=6204696 CPUtime=818.44 cores=1,3,5,7
/proc/30766/task/30795/stat : 30795 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 609381 0 0 0 81629 215 0 0 25 0 5 0 595849737 6353608704 1486835 33554432000 4194304 4338100 140736497524768 18446744073709551615 231894886126 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30766/tid=30796] ppid=30764 vsize=6204696 CPUtime=819.66 cores=1,3,5,7
/proc/30766/task/30796/stat : 30796 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 184207 0 0 0 79766 2200 0 0 25 0 5 0 595849737 6353608704 1486835 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254418 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30766/tid=30797] ppid=30764 vsize=6204696 CPUtime=819.83 cores=1,3,5,7
/proc/30766/task/30797/stat : 30797 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 476231 0 0 0 81881 102 0 0 25 0 5 0 595849737 6353608704 1486835 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30766/tid=30798] ppid=30764 vsize=6204696 CPUtime=819.81 cores=1,3,5,7
/proc/30766/task/30798/stat : 30798 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 172118 0 0 0 79824 2157 0 0 25 0 5 0 595849737 6353608704 1486835 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3279.41
Current children cumulated vsize (KiB) 6204696

[startup+882.301 s]
/proc/loadavg: 8.14 8.11 8.05 9/186 30835
/proc/meminfo: memFree=9542700/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=6204696 CPUtime=3519.07 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) S 30764 30766 30360 0 -1 4202496 1492374 0 0 0 346920 4987 0 0 25 0 5 0 595849574 6353608704 1486839 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 1551174 1486839 176 36 0 1549079 0
[pid=30766/tid=30795] ppid=30764 vsize=6204696 CPUtime=878.44 cores=1,3,5,7
/proc/30766/task/30795/stat : 30795 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 609381 0 0 0 87625 219 0 0 25 0 5 0 595849737 6353608704 1486839 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254285 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30766/tid=30796] ppid=30764 vsize=6204696 CPUtime=879.63 cores=1,3,5,7
/proc/30766/task/30796/stat : 30796 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 184208 0 0 0 85613 2350 0 0 25 0 5 0 595849737 6353608704 1486839 33554432000 4194304 4338100 140736497524768 18446744073709551615 231894886138 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30766/tid=30797] ppid=30764 vsize=6204696 CPUtime=879.83 cores=1,3,5,7
/proc/30766/task/30797/stat : 30797 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 476232 0 0 0 87881 102 0 0 25 0 5 0 595849737 6353608704 1486839 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30766/tid=30798] ppid=30764 vsize=6204696 CPUtime=879.52 cores=1,3,5,7
/proc/30766/task/30798/stat : 30798 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 172120 0 0 0 85644 2308 0 0 25 0 5 0 595849737 6353608704 1486839 33554432000 4194304 4338100 140736497524768 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3519.07
Current children cumulated vsize (KiB) 6204696

[startup+942.3 s]
/proc/loadavg: 7.93 8.06 8.03 9/186 30836
/proc/meminfo: memFree=9179372/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=6450456 CPUtime=3758.66 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) S 30764 30766 30360 0 -1 4202496 1511460 0 0 0 370581 5285 0 0 25 0 5 0 595849574 6605266944 1505925 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 1612614 1505925 176 36 0 1610519 0
[pid=30766/tid=30795] ppid=30764 vsize=6450456 CPUtime=938.41 cores=1,3,5,7
/proc/30766/task/30795/stat : 30795 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 614626 0 0 0 93616 225 0 0 25 0 5 0 595849737 6605266944 1505925 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30766/tid=30796] ppid=30764 vsize=6450456 CPUtime=939.29 cores=1,3,5,7
/proc/30766/task/30796/stat : 30796 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 191102 0 0 0 91431 2498 0 0 25 0 5 0 595849737 6605266944 1505925 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907303118 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30766/tid=30797] ppid=30764 vsize=6450456 CPUtime=939.82 cores=1,3,5,7
/proc/30766/task/30797/stat : 30797 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 477612 0 0 0 93880 102 0 0 25 0 5 0 595849737 6605266944 1505925 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30766/tid=30798] ppid=30764 vsize=6450456 CPUtime=939.48 cores=1,3,5,7
/proc/30766/task/30798/stat : 30798 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 177687 0 0 0 91496 2452 0 0 25 0 5 0 595849737 6605266944 1505925 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907303118 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3758.66
Current children cumulated vsize (KiB) 6450456

[startup+1002.3 s]
/proc/loadavg: 8.04 8.06 8.03 9/186 30838
/proc/meminfo: memFree=8856812/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=6450456 CPUtime=3998.33 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) S 30764 30766 30360 0 -1 4202496 1511463 0 0 0 394294 5539 0 0 25 0 5 0 595849574 6605266944 1505928 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 1612614 1505928 176 36 0 1610519 0
[pid=30766/tid=30795] ppid=30764 vsize=6450456 CPUtime=998.42 cores=1,3,5,7
/proc/30766/task/30795/stat : 30795 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 614628 0 0 0 99614 228 0 0 25 0 5 0 595849737 6605266944 1505928 33554432000 4194304 4338100 140736497524768 18446744073709551615 231894885777 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30766/tid=30796] ppid=30764 vsize=6450456 CPUtime=998.96 cores=1,3,5,7
/proc/30766/task/30796/stat : 30796 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 191102 0 0 0 97273 2623 0 0 25 0 5 0 595849737 6605266944 1505928 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30766/tid=30797] ppid=30764 vsize=6450456 CPUtime=999.83 cores=1,3,5,7
/proc/30766/task/30797/stat : 30797 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 477613 0 0 0 99880 103 0 0 25 0 5 0 595849737 6605266944 1505928 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30766/tid=30798] ppid=30764 vsize=6450456 CPUtime=999.48 cores=1,3,5,7
/proc/30766/task/30798/stat : 30798 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 177687 0 0 0 97370 2578 0 0 25 0 5 0 595849737 6605266944 1505928 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3998.33
Current children cumulated vsize (KiB) 6450456

[startup+1062.3 s]
/proc/loadavg: 7.84 8.00 8.01 9/186 30839
/proc/meminfo: memFree=8510824/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=6506760 CPUtime=4238.01 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) S 30764 30766 30360 0 -1 4202496 1526154 0 0 0 417996 5805 0 0 25 0 5 0 595849574 6662922240 1520619 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 1626690 1520619 176 36 0 1624595 0
[pid=30766/tid=30795] ppid=30764 vsize=6506760 CPUtime=1058.4 cores=1,3,5,7
/proc/30766/task/30795/stat : 30795 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 620552 0 0 0 105608 232 0 0 25 0 5 0 595849737 6662922240 1520619 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30766/tid=30796] ppid=30764 vsize=6506760 CPUtime=1058.63 cores=1,3,5,7
/proc/30766/task/30796/stat : 30796 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 191103 0 0 0 103111 2752 0 0 25 0 5 0 595849737 6662922240 1520619 33554432000 4194304 4338100 140736497524768 18446744073709551615 231895186987 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30766/tid=30797] ppid=30764 vsize=6506760 CPUtime=1059.82 cores=1,3,5,7
/proc/30766/task/30797/stat : 30797 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 486379 0 0 0 105877 105 0 0 25 0 5 0 595849737 6662922240 1520619 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254254 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30766/tid=30798] ppid=30764 vsize=6506760 CPUtime=1059.49 cores=1,3,5,7
/proc/30766/task/30798/stat : 30798 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 177687 0 0 0 103242 2707 0 0 25 0 5 0 595849737 6662922240 1520619 33554432000 4194304 4338100 140736497524768 18446744073709551615 231894885470 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4238.01
Current children cumulated vsize (KiB) 6506760

[startup+1122.3 s]
/proc/loadavg: 8.00 8.01 8.01 9/186 30841
/proc/meminfo: memFree=8106576/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=6660032 CPUtime=4477.62 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) S 30764 30766 30360 0 -1 4202496 1564334 0 0 0 441702 6060 0 0 25 0 5 0 595849574 6819872768 1558799 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 1665008 1558799 176 36 0 1662913 0
[pid=30766/tid=30795] ppid=30764 vsize=6660032 CPUtime=1118.4 cores=1,3,5,7
/proc/30766/task/30795/stat : 30795 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 644733 0 0 0 111597 243 0 0 25 0 5 0 595849737 6819872768 1558799 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30766/tid=30796] ppid=30764 vsize=6660032 CPUtime=1118.56 cores=1,3,5,7
/proc/30766/task/30796/stat : 30796 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 191105 0 0 0 108984 2872 0 0 25 0 5 0 595849737 6819872768 1558799 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30766/tid=30797] ppid=30764 vsize=6660032 CPUtime=1119.55 cores=1,3,5,7
/proc/30766/task/30797/stat : 30797 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 500375 0 0 0 111847 108 0 0 25 0 5 0 595849737 6819872768 1558799 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30766/tid=30798] ppid=30764 vsize=6660032 CPUtime=1119.47 cores=1,3,5,7
/proc/30766/task/30798/stat : 30798 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 177688 0 0 0 109118 2829 0 0 25 0 5 0 595849737 6819872768 1558799 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4477.62
Current children cumulated vsize (KiB) 6660032

[startup+1182.3 s]
/proc/loadavg: 8.10 8.04 8.02 9/186 30843
/proc/meminfo: memFree=7928744/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=6813304 CPUtime=4717.26 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) S 30764 30766 30360 0 -1 4202496 1603046 0 0 0 465393 6333 0 0 25 0 5 0 595849574 6976823296 1597511 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 1703326 1597511 176 36 0 1701231 0
[pid=30766/tid=30795] ppid=30764 vsize=6813304 CPUtime=1178.38 cores=1,3,5,7
/proc/30766/task/30795/stat : 30795 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 666488 0 0 0 117586 252 0 0 25 0 5 0 595849737 6976823296 1597511 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30766/tid=30796] ppid=30764 vsize=6813304 CPUtime=1178.55 cores=1,3,5,7
/proc/30766/task/30796/stat : 30796 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 191109 0 0 0 114857 2998 0 0 25 0 5 0 595849737 6976823296 1597511 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30766/tid=30797] ppid=30764 vsize=6813304 CPUtime=1179.2 cores=1,3,5,7
/proc/30766/task/30797/stat : 30797 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 517328 0 0 0 117808 112 0 0 25 0 5 0 595849737 6976823296 1597511 33554432000 4194304 4338100 140736497524768 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30766/tid=30798] ppid=30764 vsize=6813304 CPUtime=1179.46 cores=1,3,5,7
/proc/30766/task/30798/stat : 30798 (ttsth-5-4) R 30764 30766 30360 0 -1 4202560 177688 0 0 0 114985 2961 0 0 25 0 5 0 595849737 6976823296 1597511 33554432000 4194304 4338100 140736497524768 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4717.26
Current children cumulated vsize (KiB) 6813304



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.1 s]
/proc/loadavg: 8.07 8.04 8.01 6/183 30843
/proc/meminfo: memFree=9849904/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=0 CPUtime=4800.1 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) Z 30764 30766 30360 0 -1 4203548 1608918 0 0 0 473578 6432 0 0 18 0 2 0 595849574 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 0 0 0 0 0 0 0
[pid=30766/tid=30796] ppid=30764 vsize=0 CPUtime=1199.35 cores=1,3,5,7
/proc/30766/task/30796/stat : 30796 (ttsth-5-4) R 30764 30766 30360 0 -1 4203588 191109 0 0 0 116883 3052 0 0 25 0 2 0 595849737 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.1
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.21 s]
/proc/loadavg: 8.07 8.04 8.01 6/183 30843
/proc/meminfo: memFree=9849904/32950928 swapFree=67111524/67111528
[pid=30766] ppid=30764 vsize=0 CPUtime=4800.2 cores=1,3,5,7
/proc/30766/stat : 30766 (ttsth-5-4) Z 30764 30766 30360 0 -1 4203548 1608918 0 0 0 473578 6442 0 0 18 0 2 0 595849574 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/30766/statm: 0 0 0 0 0 0 0
[pid=30766/tid=30796] ppid=30764 vsize=0 CPUtime=1199.45 cores=1,3,5,7
/proc/30766/task/30796/stat : 30796 (ttsth-5-4) R 30764 30766 30360 0 -1 4203588 191109 0 0 0 116883 3062 0 0 25 0 2 0 595849737 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.2
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1203.26
CPU time (s): 4800.26
CPU user time (s): 4735.78
CPU system time (s): 64.4762
CPU usage (%): 398.937
Max. virtual memory (cumulated for all children) (KiB): 6835200

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4735.78
system time used= 64.4762
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1608918
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4543
involuntary context switches= 33102

runsolver used 2.07668 second user time and 4.50331 second system time

The end

Launcher Data 

Begin job on node114 at 2011-04-29 07:49:49
IDJOB=3259670
IDBENCH=71605
IDSOLVER=1654
FILE ID=node114/3259670-1304056189
RUNJOBID= node114-1304054983-30675
PBS_JOBID= 13170413
Free space on /tmp= 71796 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/forced/rbsat-v2640c305320gyes10.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3259670-1304056189/watcher-3259670-1304056189 -o /tmp/evaluation-result-3259670-1304056189/solver-3259670-1304056189 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259670-1304056189.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 98bac4f2d533fe7abb12552567f9e1aa
RANDOM SEED=929096166

node114.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      20506468 kB
Buffers:       2255960 kB
Cached:        9543324 kB
SwapCached:          4 kB
Active:        3406644 kB
Inactive:      8458680 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      20506468 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:            3904 kB
Writeback:           0 kB
AnonPages:       65612 kB
Mapped:          15076 kB
Slab:           516056 kB
PageTables:       4672 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   220952 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71796 MiB
End job on node114 at 2011-04-29 08:09:54