Trace number 3259230

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.27 1203.16

General information on the benchmark

NameCRAFTED/rbsat/crafted/forced/
rbsat-v760c43649gyes5.cnf
MD5SUMe7d4dc25f0d8e558fb091b1e566deab7
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.32395
Satisfiable
(Un)Satisfiability was proved
Number of variables760
Number of clauses43649
Sum of the clauses size87978
Maximum clause length19
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 243609
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 540

Solver Data 

0.00/0.02	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259230-1304054568.cnf 
0.00/0.02	c Using SATTIMEOUT of 4800
0.00/0.02	c Using SATRAM of 15500
0.00/0.02	c Size of subset node = 32
0.00/0.02	c Variables = 760
0.00/0.02	c Clauses = 43649
0.00/0.02	c Literals = 87978
0.00/0.02	c Reduced size:
0.00/0.02	c Variables = 760
0.00/0.02	c Clauses = 43649
0.00/0.02	c Literals = 87978
0.00/0.02	c a2
0.00/0.05	c a1
0.00/0.09	c Independent sub-problems = 1
0.00/0.09	c Sub-problem with 760 variables and 43649 clauses
0.00/0.09	c using permutation method 4
0.00/0.09	c count = 1
0.00/0.09	c Ordering finished 0.080 seconds
0.00/0.09	c mapped 0.090 seconds
0.00/0.09	c permuted 0.090 seconds
0.00/0.09	c about to sort
0.09/0.10	c sorted
0.09/0.18	c clauses now 43649
0.09/0.18	c init finished 0.100 seconds
0.09/0.18	c tree built 0.180 seconds
0.09/0.18	c 
0.09/0.18	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.09/0.18	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.09/0.18	c ................................ | ..........                    |                                  0.2         0.0             
0.09/0.19	c ................................ | ..........                  1 |                                  0.2         5.6             
0.09/0.19	c ................................ | ..........                  2 |                                  0.2        11.1             
0.09/0.19	c ................................ | ..........                  3 |                                  0.2        16.7             
0.09/0.19	c ................................ | ..........                  4 |                                  0.2        22.2             
0.09/0.19	c ................................ | ..........                  5 |                                  0.2        27.8             
0.09/0.19	c ................................ | ..........               1000 | 9            141                 0.2      5263.2         8772
95.67/24.18	c ................................ | ..........            3595000 | 109919       699581             96.0     37447.9     18751061
191.87/48.23	c ................................ | ..........            6484000 | 202359       1224543           192.0     33765.6     28276909
287.70/72.28	c ................................ | ..........            8963000 | 273033       1720785           288.0     31120.4     39507939
383.88/96.33	c ................................ | ..........           11375000 | 330712       2208947           384.0     29621.6     46095884
479.71/120.37	c ................................ | *.........           13663000 | 390068       2645716           480.0     28464.0     54887662
575.93/144.41	c ................................ | *.........           15807000 | 450903       3027407           576.0     27442.7     63366821
671.74/168.47	c ................................ | *.........           17905000 | 512557       3389115           672.0     26643.2     69850553
767.92/192.52	c ................................ | *.........           19970000 | 575656       3760173           768.0     26002.6     77717049
863.67/216.60	c ................................ | *.........           21720000 | 629550       4092587           864.1     25136.9     78084762
959.82/240.64	c ................................ | **........           23499000 | 677567       4403400           960.0     24478.1     81855969
1055.61/264.69	c ................................ | **........           25274000 | 728338       4717844          1056.0     23933.7     84004097
1151.82/288.75	c ................................ | **........           26906000 | 776023       5009448          1152.0     23355.3     90441867
1248.02/312.81	c ................................ | **........           28453000 | 822379       5297536          1248.1     22797.4     95486157
1343.78/336.87	c ................................ | **........           30083000 | 866755       5572718          1344.1     22382.2    100178785
1439.98/360.92	c ................................ | ***.......           31605000 | 906453       5802599          1440.1     21946.8    104914164
1535.73/384.98	c ................................ | ***.......           32957000 | 941799       6036366          1536.0     21455.8    104914164
1631.93/409.01	c ................................ | ***.......           34271000 | 976798       6273493          1632.0     20999.4    104914164
1727.72/433.07	c ................................ | ***.......           35591000 | 1013780      6510065          1728.0     20596.4    109493831
1823.95/457.11	c ................................ | ***.......           36976000 | 1051934      6766603          1824.0     20271.7    114493575
1919.66/481.19	c ................................ | ****......           38272000 | 1090938      7030690          1920.0     19933.2    116815875
2015.89/505.23	c ................................ | ****......           39536000 | 1130264      7313774          2016.0     19610.8    120371715
2111.67/529.28	c ................................ | ****......           40799000 | 1168493      7604548          2112.0     19317.7    125758885
2207.85/553.34	c ................................ | ****......           42242000 | 1207371      7881308          2208.0     19131.3    130868708
2303.93/577.42	c ................................ | ****......           43988000 | 1253931      8170376          2304.0     19092.0    135831897
2400.00/601.50	c ................................ | *****.....           45488000 | 1300035      8403607          2400.0     18953.0    137974673
2495.68/625.59	c ................................ | *****.....           47131000 | 1348660      8662975          2496.0     18882.4    139561793
2591.91/649.63	c ................................ | *****.....           48572000 | 1389400      8892058          2592.0     18738.9    139953710
2687.66/673.70	c ................................ | *****.....           50050000 | 1430949      9144881          2688.1     18619.4    146085803
2783.80/697.75	c ................................ | *****.....           51415000 | 1469795      9389789          2784.0     18467.8    151633477
2880.33/721.97	c ................................ | ******....           52810000 | 1514211      9643929          2880.6     18332.9    157111517
2975.66/745.89	c ................................ | ******....           54059000 | 1552876      9839203          2976.0     18164.9    157323627
3071.85/769.95	c ................................ | ******....           55273000 | 1590643      10079330         3072.1     17992.2    157835088
3168.02/794.00	c ................................ | ******....           56495000 | 1630230      10319160         3168.1     17832.7    162708475
3263.76/818.06	c ................................ | ******....           57635000 | 1663333      10543118         3264.0     17657.6    164069609
3359.91/842.13	c ................................ | *******...           58849000 | 1699554      10758985         3360.0     17514.4    164069609
3455.70/866.19	c ................................ | *******...           60204000 | 1738070      10975759         3456.1     17419.8    166112716
3551.93/890.22	c ................................ | *******...           61496000 | 1777059      11195924         3552.0     17313.0    171046420
3647.69/914.29	c ................................ | *******...           62953000 | 1821276      11425881         3648.1     17256.6    173018504
3743.84/938.35	c ................................ | *******...           64234000 | 1864958      11628343         3744.0     17156.4    175623311
3839.65/962.38	c ................................ | ********..           65438000 | 1910220      11821050         3840.0     17041.1    175623311
3935.78/986.47	c ................................ | ********..           66702000 | 1959149      12014002         3936.1     16946.4    176968501
4031.98/1010.51	c ................................ | ********..           67789000 | 2000963      12199132         4032.0     16812.6    177934105
4127.77/1034.56	c ................................ | ********..           68965000 | 2039472      12390688         4128.0     16706.5    179753588
4223.96/1058.62	c ................................ | ********..           70214000 | 2080543      12620058         4224.0     16622.5    184446337
4319.71/1082.68	c ................................ | *********.           71317000 | 2121500      12834446         4320.1     16508.3    188116323
4415.89/1106.74	c ................................ | *********.           72385000 | 2156595      13068524         4416.1     16391.3    188116323
4511.73/1130.78	c ................................ | *********.           73430000 | 2196090      13279384         4512.1     16274.2    189353005
4607.90/1154.84	c ................................ | *********.           74613000 | 2234670      13482343         4608.1     16191.8    191334598
4703.66/1178.89	c ................................ | *********.           75778000 | 2271023      13677192         4704.0     16109.1    191538419
4799.91/1202.93	c CPU limit (4800) reached
4799.91/1202.93	c Time = 4800.03 seconds
4799.91/1202.93	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3259230-1304054568/watcher-3259230-1304054568 -o /tmp/evaluation-result-3259230-1304054568/solver-3259230-1304054568 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259230-1304054568.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.12 8.10 8.02 1/175 30780
/proc/meminfo: memFree=15163376/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=9760 CPUtime=0 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) R 30778 30780 30368 0 -1 4202496 496 0 0 0 0 0 0 0 25 0 1 0 517056459 9994240 438 33554432000 4194304 4338100 140734917115840 18446744073709551615 274473628357 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/30780/statm: 2440 439 147 36 0 345 0

[startup+0.011347 s]
/proc/loadavg: 8.12 8.10 8.02 1/175 30780
/proc/meminfo: memFree=15163376/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=10900 CPUtime=0 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) R 30778 30780 30368 0 -1 4202496 772 0 0 0 0 0 0 0 25 0 1 0 517056459 11161600 714 33554432000 4194304 4338100 140734917115840 18446744073709551615 274473628357 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/30780/statm: 2725 715 148 36 0 630 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 10900

[startup+0.100335 s]
/proc/loadavg: 8.12 8.10 8.02 1/175 30780
/proc/meminfo: memFree=15163376/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=18136 CPUtime=0.09 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) R 30778 30780 30368 0 -1 4202496 3489 0 0 0 9 0 0 0 25 0 1 0 517056459 18571264 2575 33554432000 4194304 4338100 140734917115840 18446744073709551615 274473625409 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/30780/statm: 4534 2575 161 36 0 2439 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 18136

[startup+0.3003 s]
/proc/loadavg: 8.12 8.10 8.02 1/175 30780
/proc/meminfo: memFree=15163376/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=159788 CPUtime=0.62 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 11030 0 0 0 60 2 0 0 25 0 5 0 517056459 163622912 10029 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 39947 10029 176 36 0 37852 0
Current children cumulated CPU time (s) 0.62
Current children cumulated vsize (KiB) 159788

[startup+0.700231 s]
/proc/loadavg: 8.12 8.10 8.02 1/175 30780
/proc/meminfo: memFree=15163376/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=241360 CPUtime=2.21 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 15937 0 0 0 217 4 0 0 25 0 5 0 517056459 247152640 14936 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 60340 14936 176 36 0 58245 0
Current children cumulated CPU time (s) 2.21
Current children cumulated vsize (KiB) 241360

[startup+1.50109 s]
/proc/loadavg: 7.55 7.98 7.98 6/182 30812
/proc/meminfo: memFree=15081696/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=282420 CPUtime=5.41 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 24317 0 0 0 533 8 0 0 25 0 5 0 517056459 289198080 23316 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 70605 23317 176 36 0 68510 0
[pid=30780/tid=30782] ppid=30778 vsize=282420 CPUtime=1.3 cores=1,3,5,7
/proc/30780/task/30782/stat : 30782 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 5193 0 0 0 129 1 0 0 20 0 5 0 517056478 289198080 23317 33554432000 4194304 4338100 140734917115840 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30780/tid=30783] ppid=30778 vsize=282420 CPUtime=1.29 cores=1,3,5,7
/proc/30780/task/30783/stat : 30783 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 3382 0 0 0 127 2 0 0 19 0 5 0 517056478 289198080 23317 33554432000 4194304 4338100 140734917115840 18446744073709551615 4239444 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=282420 CPUtime=1.29 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 5001 0 0 0 129 0 0 0 25 0 5 0 517056478 289198080 23317 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30780/tid=30785] ppid=30778 vsize=282420 CPUtime=1.3 cores=1,3,5,7
/proc/30780/task/30785/stat : 30785 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 1747 0 0 0 128 2 0 0 25 0 5 0 517056478 289198080 23318 33554432000 4194304 4338100 140734917115840 18446744073709551615 274473625341 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 5.41
Current children cumulated vsize (KiB) 282420

[startup+3.10081 s]
/proc/loadavg: 7.55 7.98 7.98 9/186 30817
/proc/meminfo: memFree=14957416/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=345508 CPUtime=11.8 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 40659 0 0 0 1163 17 0 0 25 0 5 0 517056459 353800192 39658 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 86377 39659 176 36 0 84282 0
[pid=30780/tid=30782] ppid=30778 vsize=345508 CPUtime=2.9 cores=1,3,5,7
/proc/30780/task/30782/stat : 30782 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 10858 0 0 0 288 2 0 0 25 0 5 0 517056478 353800192 39659 33554432000 4194304 4338100 140734917115840 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30780/tid=30783] ppid=30778 vsize=345508 CPUtime=2.9 cores=1,3,5,7
/proc/30780/task/30783/stat : 30783 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 6475 0 0 0 284 6 0 0 24 0 5 0 517056478 353800192 39659 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=345508 CPUtime=2.89 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 10369 0 0 0 288 1 0 0 25 0 5 0 517056478 353800192 39659 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254141 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30780/tid=30785] ppid=30778 vsize=345508 CPUtime=2.9 cores=1,3,5,7
/proc/30780/task/30785/stat : 30785 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 3963 0 0 0 284 6 0 0 25 0 5 0 517056478 353800192 39659 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 11.8
Current children cumulated vsize (KiB) 345508

[startup+6.30024 s]
/proc/loadavg: 7.67 7.99 7.98 10/186 30817
/proc/meminfo: memFree=14729380/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=441192 CPUtime=24.58 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 64729 0 0 0 2419 39 0 0 25 0 5 0 517056459 451780608 63728 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 110298 63728 176 36 0 108203 0
[pid=30780/tid=30782] ppid=30778 vsize=441192 CPUtime=6.1 cores=1,3,5,7
/proc/30780/task/30782/stat : 30782 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 19683 0 0 0 606 4 0 0 25 0 5 0 517056478 451780608 63728 33554432000 4194304 4338100 140734917115840 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30780/tid=30783] ppid=30778 vsize=441192 CPUtime=6.1 cores=1,3,5,7
/proc/30780/task/30783/stat : 30783 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 10187 0 0 0 595 15 0 0 25 0 5 0 517056478 451780608 63728 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=441192 CPUtime=6.07 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 18845 0 0 0 604 3 0 0 25 0 5 0 517056478 451780608 63728 33554432000 4194304 4338100 140734917115840 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30780/tid=30785] ppid=30778 vsize=441192 CPUtime=6.1 cores=1,3,5,7
/proc/30780/task/30785/stat : 30785 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 7019 0 0 0 595 15 0 0 25 0 5 0 517056478 451780608 63728 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 24.58
Current children cumulated vsize (KiB) 441192

[startup+12.7011 s]
/proc/loadavg: 7.77 8.01 7.99 10/186 30817
/proc/meminfo: memFree=14333076/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=625744 CPUtime=50.15 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 111228 0 0 0 4928 87 0 0 25 0 5 0 517056459 640761856 110227 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 156436 110227 176 36 0 154341 0
[pid=30780/tid=30782] ppid=30778 vsize=625744 CPUtime=12.5 cores=1,3,5,7
/proc/30780/task/30782/stat : 30782 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 35621 0 0 0 1242 8 0 0 25 0 5 0 517056478 640761856 110227 33554432000 4194304 4338100 140734917115840 18446744073709551615 274473965627 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30780/tid=30783] ppid=30778 vsize=625744 CPUtime=12.5 cores=1,3,5,7
/proc/30780/task/30783/stat : 30783 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 17486 0 0 0 1214 36 0 0 25 0 5 0 517056478 640761856 110228 33554432000 4194304 4338100 140734917115840 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=625744 CPUtime=12.43 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 35393 0 0 0 1237 6 0 0 25 0 5 0 517056478 640761856 110228 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30780/tid=30785] ppid=30778 vsize=625744 CPUtime=12.49 cores=1,3,5,7
/proc/30780/task/30785/stat : 30785 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 13733 0 0 0 1215 34 0 0 25 0 5 0 517056478 640761856 110228 33554432000 4194304 4338100 140734917115840 18446744073709551615 4253954 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 50.15
Current children cumulated vsize (KiB) 625744

[startup+25.5009 s]
/proc/loadavg: 7.89 8.03 7.99 9/186 30817
/proc/meminfo: memFree=13779048/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=853972 CPUtime=101.24 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 167454 0 0 0 9949 175 0 0 25 0 5 0 517056459 874467328 166453 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 213493 166454 176 36 0 211398 0
[pid=30780/tid=30782] ppid=30778 vsize=853972 CPUtime=25.29 cores=1,3,5,7
/proc/30780/task/30782/stat : 30782 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 62610 0 0 0 2514 15 0 0 25 0 5 0 517056478 874467328 166454 33554432000 4194304 4338100 140734917115840 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30780/tid=30783] ppid=30778 vsize=853972 CPUtime=25.3 cores=1,3,5,7
/proc/30780/task/30783/stat : 30783 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 23676 0 0 0 2457 73 0 0 25 0 5 0 517056478 874467328 166454 33554432000 4194304 4338100 140734917115840 18446744073709551615 274473362878 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=853972 CPUtime=25.21 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 53433 0 0 0 2510 11 0 0 25 0 5 0 517056478 874467328 166454 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30780/tid=30785] ppid=30778 vsize=853972 CPUtime=25.23 cores=1,3,5,7
/proc/30780/task/30785/stat : 30785 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 18741 0 0 0 2449 74 0 0 25 0 5 0 517056478 874467328 166454 33554432000 4194304 4338100 140734917115840 18446744073709551615 274473363175 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.24
Current children cumulated vsize (KiB) 853972

[startup+51.1004 s]
/proc/loadavg: 8.00 8.04 8.00 9/186 30818
/proc/meminfo: memFree=12977128/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=1209796 CPUtime=203.45 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 255793 0 0 0 19984 361 0 0 25 0 5 0 517056459 1238831104 254792 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 302449 254794 176 36 0 300354 0
[pid=30780/tid=30782] ppid=30778 vsize=1209796 CPUtime=50.87 cores=1,3,5,7
/proc/30780/task/30782/stat : 30782 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 104983 0 0 0 5058 29 0 0 25 0 5 0 517056478 1238831104 254794 33554432000 4194304 4338100 140734917115840 18446744073709551615 274473363323 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30780/tid=30783] ppid=30778 vsize=1209796 CPUtime=50.88 cores=1,3,5,7
/proc/30780/task/30783/stat : 30783 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 27814 0 0 0 4933 155 0 0 25 0 5 0 517056478 1238831104 254794 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=1209796 CPUtime=50.78 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 92287 0 0 0 5060 18 0 0 25 0 5 0 517056478 1238831104 254794 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30780/tid=30785] ppid=30778 vsize=1209796 CPUtime=50.69 cores=1,3,5,7
/proc/30780/task/30785/stat : 30785 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 21716 0 0 0 4913 156 0 0 25 0 5 0 517056478 1238831104 254794 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 203.45
Current children cumulated vsize (KiB) 1209796

[startup+102.306 s]
/proc/loadavg: 8.08 8.05 8.00 9/186 30819
/proc/meminfo: memFree=11958964/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=1801988 CPUtime=407.85 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 402878 0 0 0 40037 748 0 0 25 0 5 0 517056459 1845235712 401877 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 450497 401877 176 36 0 448402 0
[pid=30780/tid=30782] ppid=30778 vsize=1801988 CPUtime=101.93 cores=1,3,5,7
/proc/30780/task/30782/stat : 30782 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 175224 0 0 0 10146 47 0 0 25 0 5 0 517056478 1845235712 401877 33554432000 4194304 4338100 140734917115840 18446744073709551615 274473362713 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30780/tid=30783] ppid=30778 vsize=1801988 CPUtime=102.06 cores=1,3,5,7
/proc/30780/task/30783/stat : 30783 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 39461 0 0 0 9870 336 0 0 25 0 5 0 517056478 1845235712 401877 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=1801988 CPUtime=101.76 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 147950 0 0 0 10146 30 0 0 25 0 5 0 517056478 1845235712 401877 33554432000 4194304 4338100 140734917115840 18446744073709551615 4253954 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30780/tid=30785] ppid=30778 vsize=1801988 CPUtime=101.86 cores=1,3,5,7
/proc/30780/task/30785/stat : 30785 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 31248 0 0 0 9854 332 0 0 25 0 5 0 517056478 1845235712 401877 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 407.85
Current children cumulated vsize (KiB) 1801988

[startup+162.301 s]
/proc/loadavg: 8.09 8.05 8.00 9/186 30821
/proc/meminfo: memFree=10835884/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=2412044 CPUtime=647.38 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 556851 0 0 0 63600 1138 0 0 25 0 5 0 517056459 2469933056 555850 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 603011 555850 176 36 0 600916 0
[pid=30780/tid=30782] ppid=30778 vsize=2412044 CPUtime=161.57 cores=1,3,5,7
/proc/30780/task/30782/stat : 30782 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 248654 0 0 0 16088 69 0 0 25 0 5 0 517056478 2469933056 555850 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30780/tid=30783] ppid=30778 vsize=2412044 CPUtime=162.03 cores=1,3,5,7
/proc/30780/task/30783/stat : 30783 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 52626 0 0 0 15686 517 0 0 25 0 5 0 517056478 2469933056 555850 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=2412044 CPUtime=161.74 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 206135 0 0 0 16131 43 0 0 25 0 5 0 517056478 2469933056 555850 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30780/tid=30785] ppid=30778 vsize=2412044 CPUtime=161.81 cores=1,3,5,7
/proc/30780/task/30785/stat : 30785 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 40441 0 0 0 15675 506 0 0 25 0 5 0 517056478 2469933056 555850 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 647.38
Current children cumulated vsize (KiB) 2412044


################
# More data... #
################

/proc/30780/task/30785/stat : 30785 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 57779 0 0 0 68274 1766 0 0 25 0 5 0 517056478 5424713728 1275449 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2802.15
Current children cumulated vsize (KiB) 5297572

[startup+762.306 s]
/proc/loadavg: 8.03 8.06 8.01 9/186 30837
/proc/meminfo: memFree=4893228/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=5601024 CPUtime=3041.49 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 1323925 0 0 0 300021 4128 0 0 25 0 5 0 517056459 5735448576 1322924 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 1400256 1322924 176 36 0 1398161 0
[pid=30780/tid=30782] ppid=30778 vsize=5601024 CPUtime=760.35 cores=1,3,5,7
/proc/30780/task/30782/stat : 30782 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 646930 0 0 0 75807 228 0 0 25 0 5 0 517056478 5735448576 1322924 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30780/tid=30783] ppid=30778 vsize=5601024 CPUtime=761.26 cores=1,3,5,7
/proc/30780/task/30783/stat : 30783 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 79522 0 0 0 74233 1893 0 0 25 0 5 0 517056478 5735448576 1322924 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=5601024 CPUtime=759.36 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 525050 0 0 0 75815 121 0 0 25 0 5 0 517056478 5735448576 1322924 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30780/tid=30785] ppid=30778 vsize=5601024 CPUtime=760.31 cores=1,3,5,7
/proc/30780/task/30785/stat : 30785 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 63428 0 0 0 74147 1884 0 0 25 0 5 0 517056478 5735448576 1322924 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3041.49
Current children cumulated vsize (KiB) 5601024

[startup+822.301 s]
/proc/loadavg: 8.01 8.05 8.00 9/186 30838
/proc/meminfo: memFree=4435412/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=5815972 CPUtime=3280.93 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 1383956 0 0 0 323663 4430 0 0 25 0 5 0 517056459 5955555328 1382955 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 1453993 1382955 176 36 0 1451898 0
[pid=30780/tid=30782] ppid=30778 vsize=5815972 CPUtime=820.26 cores=1,3,5,7
/proc/30780/task/30782/stat : 30782 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 675963 0 0 0 81784 242 0 0 25 0 5 0 517056478 5955555328 1382955 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254285 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30780/tid=30783] ppid=30778 vsize=5815972 CPUtime=821.19 cores=1,3,5,7
/proc/30780/task/30783/stat : 30783 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 82814 0 0 0 80085 2034 0 0 25 0 5 0 517056478 5955555328 1382955 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=5815972 CPUtime=819 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 549740 0 0 0 81773 127 0 0 25 0 5 0 517056478 5955555328 1382955 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30780/tid=30785] ppid=30778 vsize=5815972 CPUtime=820.25 cores=1,3,5,7
/proc/30780/task/30785/stat : 30785 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 66444 0 0 0 80001 2024 0 0 25 0 5 0 517056478 5955555328 1382955 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3280.93
Current children cumulated vsize (KiB) 5815972

[startup+882.301 s]
/proc/loadavg: 8.07 8.06 8.00 9/186 30840
/proc/meminfo: memFree=4176128/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=6011888 CPUtime=3520.37 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 1436424 0 0 0 347341 4696 0 0 25 0 5 0 517056459 6156173312 1435423 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 1502972 1435423 176 36 0 1500877 0
[pid=30780/tid=30782] ppid=30778 vsize=6011888 CPUtime=880.17 cores=1,3,5,7
/proc/30780/task/30782/stat : 30782 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 702471 0 0 0 87764 253 0 0 25 0 5 0 517056478 6156173312 1435423 33554432000 4194304 4338100 140734917115840 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30780/tid=30783] ppid=30778 vsize=6011888 CPUtime=881.16 cores=1,3,5,7
/proc/30780/task/30783/stat : 30783 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 84293 0 0 0 85958 2158 0 0 25 0 5 0 517056478 6156173312 1435423 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=6011888 CPUtime=878.61 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 573247 0 0 0 87728 133 0 0 25 0 5 0 517056478 6156173312 1435423 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30780/tid=30785] ppid=30778 vsize=6011888 CPUtime=880.2 cores=1,3,5,7
/proc/30780/task/30785/stat : 30785 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 67418 0 0 0 85871 2149 0 0 25 0 5 0 517056478 6156173312 1435423 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3520.37
Current children cumulated vsize (KiB) 6011888

[startup+942.301 s]
/proc/loadavg: 8.09 8.06 8.00 9/186 30841
/proc/meminfo: memFree=3798532/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=6179916 CPUtime=3759.79 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 1481867 0 0 0 371048 4931 0 0 25 0 5 0 517056459 6328233984 1480866 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 1544979 1480866 176 36 0 1542884 0
[pid=30780/tid=30782] ppid=30778 vsize=6179916 CPUtime=940.11 cores=1,3,5,7
/proc/30780/task/30782/stat : 30782 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 726489 0 0 0 93747 264 0 0 25 0 5 0 517056478 6328233984 1480866 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30780/tid=30783] ppid=30778 vsize=6179916 CPUtime=940.83 cores=1,3,5,7
/proc/30780/task/30783/stat : 30783 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 85659 0 0 0 91813 2270 0 0 25 0 5 0 517056478 6328233984 1480866 33554432000 4194304 4338100 140734917115840 18446744073709551615 4253966 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=6179916 CPUtime=938.49 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 591424 0 0 0 93711 138 0 0 25 0 5 0 517056478 6328233984 1480866 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30780/tid=30785] ppid=30778 vsize=6179916 CPUtime=940.15 cores=1,3,5,7
/proc/30780/task/30785/stat : 30785 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 69300 0 0 0 91759 2256 0 0 25 0 5 0 517056478 6328233984 1480866 33554432000 4194304 4338100 140734917115840 18446744073709551615 4253966 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3759.79
Current children cumulated vsize (KiB) 6179916

[startup+1002.3 s]
/proc/loadavg: 8.17 8.08 8.01 9/186 30843
/proc/meminfo: memFree=3310468/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=6256704 CPUtime=3999.28 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 1508232 0 0 0 394784 5144 0 0 25 0 5 0 517056459 6406864896 1507231 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 1564176 1507231 176 36 0 1562081 0
[pid=30780/tid=30782] ppid=30778 vsize=6256704 CPUtime=1000.04 cores=1,3,5,7
/proc/30780/task/30782/stat : 30782 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 740770 0 0 0 99730 274 0 0 25 0 5 0 517056478 6406864896 1507231 33554432000 4194304 4338100 140734917115840 18446744073709551615 4256087 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30780/tid=30783] ppid=30778 vsize=6256704 CPUtime=1000.47 cores=1,3,5,7
/proc/30780/task/30783/stat : 30783 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 88874 0 0 0 97676 2371 0 0 25 0 5 0 517056478 6406864896 1507231 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=6256704 CPUtime=998.44 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 596004 0 0 0 99705 139 0 0 25 0 5 0 517056478 6406864896 1507231 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30780/tid=30785] ppid=30778 vsize=6256704 CPUtime=1000.11 cores=1,3,5,7
/proc/30780/task/30785/stat : 30785 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 73589 0 0 0 97654 2357 0 0 25 0 5 0 517056478 6406864896 1507231 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3999.28
Current children cumulated vsize (KiB) 6256704

[startup+1062.3 s]
/proc/loadavg: 8.19 8.09 8.01 9/186 30845
/proc/meminfo: memFree=2969080/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=6490420 CPUtime=4238.73 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 1570486 0 0 0 418495 5378 0 0 25 0 5 0 517056459 6646190080 1569485 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 1622605 1569485 176 36 0 1620510 0
[pid=30780/tid=30782] ppid=30778 vsize=6490420 CPUtime=1059.97 cores=1,3,5,7
/proc/30780/task/30782/stat : 30782 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 769163 0 0 0 105712 285 0 0 25 0 5 0 517056478 6646190080 1569485 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30780/tid=30783] ppid=30778 vsize=6490420 CPUtime=1060.09 cores=1,3,5,7
/proc/30780/task/30783/stat : 30783 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 90507 0 0 0 103530 2479 0 0 25 0 5 0 517056478 6646190080 1569485 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=6490420 CPUtime=1058.4 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 627076 0 0 0 105692 148 0 0 25 0 5 0 517056478 6646190080 1569485 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30780/tid=30785] ppid=30778 vsize=6490420 CPUtime=1060.06 cores=1,3,5,7
/proc/30780/task/30785/stat : 30785 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 74745 0 0 0 103542 2464 0 0 25 0 5 0 517056478 6646190080 1569485 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4238.73
Current children cumulated vsize (KiB) 6490420

[startup+1122.3 s]
/proc/loadavg: 8.18 8.10 8.02 9/186 30846
/proc/meminfo: memFree=2652120/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=6592628 CPUtime=4478.19 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 1606030 0 0 0 442210 5609 0 0 25 0 5 0 517056459 6750851072 1605029 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 1648157 1605029 176 36 0 1646062 0
[pid=30780/tid=30782] ppid=30778 vsize=6592628 CPUtime=1119.87 cores=1,3,5,7
/proc/30780/task/30782/stat : 30782 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 778900 0 0 0 111696 291 0 0 25 0 5 0 517056478 6750851072 1605029 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30780/tid=30783] ppid=30778 vsize=6592628 CPUtime=1119.86 cores=1,3,5,7
/proc/30780/task/30783/stat : 30783 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 95244 0 0 0 109396 2590 0 0 25 0 5 0 517056478 6750851072 1605029 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=6592628 CPUtime=1118.21 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 643825 0 0 0 111668 153 0 0 25 0 5 0 517056478 6750851072 1605029 33554432000 4194304 4338100 140734917115840 18446744073709551615 274473363296 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30780/tid=30785] ppid=30778 vsize=6592628 CPUtime=1120.03 cores=1,3,5,7
/proc/30780/task/30785/stat : 30785 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 79066 0 0 0 109430 2573 0 0 25 0 5 0 517056478 6750851072 1605029 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4478.19
Current children cumulated vsize (KiB) 6592628

[startup+1182.3 s]
/proc/loadavg: 8.17 8.11 8.02 9/186 30848
/proc/meminfo: memFree=2519680/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=6694572 CPUtime=4717.64 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) S 30778 30780 30368 0 -1 4202496 1637407 0 0 0 465924 5840 0 0 25 0 5 0 517056459 6855241728 1636406 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 1673643 1636406 176 36 0 1671548 0
[pid=30780/tid=30782] ppid=30778 vsize=6694572 CPUtime=1179.49 cores=1,3,5,7
/proc/30780/task/30782/stat : 30782 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 785646 0 0 0 117652 297 0 0 25 0 5 0 517056478 6855241728 1636406 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=30780/tid=30783] ppid=30778 vsize=6694572 CPUtime=1179.81 cores=1,3,5,7
/proc/30780/task/30783/stat : 30783 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 97987 0 0 0 115286 2695 0 0 25 0 5 0 517056478 6855241728 1636406 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=6694572 CPUtime=1178.17 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 662666 0 0 0 117658 159 0 0 25 0 5 0 517056478 6855241728 1636406 33554432000 4194304 4338100 140734917115840 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=30780/tid=30785] ppid=30778 vsize=6694572 CPUtime=1179.95 cores=1,3,5,7
/proc/30780/task/30785/stat : 30785 (ttsth-5-4) R 30778 30780 30368 0 -1 4202560 82113 0 0 0 115308 2687 0 0 25 0 5 0 517056478 6855241728 1636406 33554432000 4194304 4338100 140734917115840 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4717.64
Current children cumulated vsize (KiB) 6694572



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203 s]
/proc/loadavg: 8.12 8.11 8.02 6/183 30849
/proc/meminfo: memFree=4031004/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=0 CPUtime=4800.11 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) Z 30778 30780 30368 0 -1 4203548 1667641 0 0 0 474077 5934 0 0 18 0 2 0 517056459 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 0 0 0 0 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=0 CPUtime=1198.88 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4203588 677367 0 0 0 119717 171 0 0 25 0 2 0 517056478 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.11
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.11 s]
/proc/loadavg: 8.12 8.11 8.02 6/183 30849
/proc/meminfo: memFree=4031004/32950928 swapFree=67111528/67111528
[pid=30780] ppid=30778 vsize=0 CPUtime=4800.21 cores=1,3,5,7
/proc/30780/stat : 30780 (ttsth-5-4) Z 30778 30780 30368 0 -1 4203548 1667641 0 0 0 474077 5944 0 0 18 0 2 0 517056459 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/30780/statm: 0 0 0 0 0 0 0
[pid=30780/tid=30784] ppid=30778 vsize=0 CPUtime=1198.98 cores=1,3,5,7
/proc/30780/task/30784/stat : 30784 (ttsth-5-4) R 30778 30780 30368 0 -1 4203588 677367 0 0 0 119717 181 0 0 25 0 2 0 517056478 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1203.16
CPU time (s): 4800.27
CPU user time (s): 4740.78
CPU system time (s): 59.499
CPU usage (%): 398.971
Max. virtual memory (cumulated for all children) (KiB): 6814096

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4740.78
system time used= 59.499
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1667641
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 9050
involuntary context switches= 31604

runsolver used 1.9917 second user time and 4.5833 second system time

The end

Launcher Data 

Begin job on node125 at 2011-04-29 07:22:48
IDJOB=3259230
IDBENCH=71526
IDSOLVER=1654
FILE ID=node125/3259230-1304054568
RUNJOBID= node125-1304053363-30689
PBS_JOBID= 13170412
Free space on /tmp= 72764 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/forced/rbsat-v760c43649gyes5.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3259230-1304054568/watcher-3259230-1304054568 -o /tmp/evaluation-result-3259230-1304054568/solver-3259230-1304054568 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259230-1304054568.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= e7d4dc25f0d8e558fb091b1e566deab7
RANDOM SEED=1598484405

node125.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5266.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      15163408 kB
Buffers:       2218276 kB
Cached:       14972732 kB
SwapCached:          0 kB
Active:        9042924 kB
Inactive:      8213748 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      15163408 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:             436 kB
Writeback:           0 kB
AnonPages:       64708 kB
Mapped:          15076 kB
Slab:           468456 kB
PageTables:       4244 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   218480 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72764 MiB
End job on node125 at 2011-04-29 07:42:53