Trace number 3259318

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.22 1203.89

General information on the benchmark

NameCRAFTED/rbsat/crafted/
forced/rbsat-v2400c266431gyes3.cnf
MD5SUMcef3d3066fec910e94448f2404c09548
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2400
Number of clauses266431
Sum of the clauses size535112
Maximum clause length32
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2266356
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 575

Solver Data 

0.09/0.18	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259318-1304054983.cnf 
0.09/0.18	c Using SATTIMEOUT of 4800
0.09/0.18	c Using SATRAM of 15500
0.09/0.18	c Size of subset node = 32
0.09/0.18	c Variables = 2400
0.09/0.18	c Clauses = 266431
0.09/0.18	c Literals = 535112
0.09/0.18	c Reduced size:
0.09/0.18	c Variables = 2400
0.09/0.18	c Clauses = 266431
0.09/0.18	c Literals = 535112
0.09/0.18	c a2
0.29/0.33	c a1
0.69/0.70	c Independent sub-problems = 1
0.69/0.70	c Sub-problem with 2400 variables and 266431 clauses
0.69/0.70	c using permutation method 4
0.69/0.70	c count = 1
0.69/0.70	c Ordering finished 0.660 seconds
0.69/0.70	c mapped 0.690 seconds
0.69/0.70	c permuted 0.690 seconds
0.69/0.70	c about to sort
0.69/0.72	c sorted
1.39/1.40	c clauses now 266431
1.39/1.40	c init finished 0.770 seconds
1.39/1.40	c tree built 1.390 seconds
1.39/1.40	c 
1.39/1.40	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
1.39/1.40	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
1.39/1.40	c ................................ | ..........                    |                                  1.4         0.0             
1.39/1.40	c ................................ | ..........                  1 |              6                   1.4         0.7             
1.39/1.40	c ................................ | ..........                  2 |              6                   1.4         1.4             
1.39/1.40	c ................................ | ..........                  3 |              6                   1.4         2.1             
1.39/1.40	c ................................ | ..........                  4 |              6                   1.4         2.9             
1.39/1.40	c ................................ | ..........                  5 |              6                   1.4         3.6             
1.39/1.40	c ................................ | ..........               1000 |              85                  1.4       714.3         1148
95.65/25.08	c ................................ | ..........            3613000 | 67399        285369             96.0     37631.5     13854481
191.90/49.13	c ................................ | ..........            6652000 | 121474       521209            192.0     34640.4     23203663
287.73/73.17	c ................................ | ..........            9338000 | 167101       720502            288.0     32422.5     27107548
383.99/97.20	c ................................ | ..........           11817000 | 210699       895608            384.0     30771.8     31817655
479.84/121.24	c ................................ | *.........           14106000 | 251461       1065779           480.0     29385.7     36714705
575.73/145.27	c ................................ | *.........           16335000 | 293174       1232876           576.0     28358.9     41325751
671.99/169.31	c ................................ | *.........           18517000 | 333470       1389236           672.0     27553.8     46030645
767.83/193.34	c ................................ | *.........           20617000 | 373100       1528721           768.0     26844.4     52635670
863.70/217.38	c ................................ | *.........           22736000 | 410664       1674949           864.0     26313.9     54326535
959.94/241.41	c ................................ | **........           24987000 | 445905       1816641           960.0     26027.9     57579155
1055.79/265.45	c ................................ | **........           27080000 | 480752       1948725          1056.0     25643.9     59818892
1152.04/289.50	c ................................ | **........           29200000 | 517779       2076726          1152.0     25346.3     61813185
1247.92/313.52	c ................................ | **........           31295000 | 552841       2198217          1248.0     25076.1     65734887
1343.72/337.57	c ................................ | **........           33270000 | 587324       2326599          1344.0     24754.5     67796159
1439.96/361.61	c ................................ | ***.......           35101000 | 617318       2443725          1440.0     24375.2     69323364
1535.80/385.65	c ................................ | ***.......           36903000 | 647075       2564958          1536.0     24025.2     69437262
1631.65/409.69	c ................................ | ***.......           38660000 | 678426       2679335          1632.0     23688.3     70510393
1727.90/433.73	c ................................ | ***.......           40362000 | 710081       2784779          1728.0     23357.4     71500639
1823.73/457.77	c ................................ | ***.......           42149000 | 742631       2895580          1824.0     23107.6     75789130
1919.61/481.80	c ................................ | ****......           43933000 | 771234       3009154          1920.0     22881.8     79769225
2015.87/505.84	c ................................ | ****......           45614000 | 797437       3123005          2016.0     22625.4     84077678
2111.70/529.87	c ................................ | ****......           47237000 | 826703       3234195          2112.0     22366.0     85666008
2207.76/553.96	c ................................ | ****......           48805000 | 855615       3339360          2208.0     22103.4     88643410
2303.66/578.09	c ................................ | ****......           50468000 | 887114       3443009          2304.0     21904.1     92416245
2399.91/602.12	c ................................ | *****.....           52061000 | 916433       3545967          2400.0     21691.8     94820148
2495.65/626.19	c ................................ | *****.....           53551000 | 944902       3643667          2496.0     21454.4     95079305
2591.89/650.24	c ................................ | *****.....           55034000 | 971192       3732642          2592.1     21231.8     95079305
2687.72/674.28	c ................................ | *****.....           56519000 | 998718       3828292          2688.0     21026.3     95079305
2783.94/698.32	c ................................ | *****.....           57892000 | 1025301      3918103          2784.0     20794.3     95079305
2879.77/722.36	c ................................ | ******....           59344000 | 1055188      4010396          2880.0     20605.3     95925666
2976.01/746.41	c ................................ | ******....           60737000 | 1082591      4100269          2976.1     20408.4     97174504
3071.85/770.43	c ................................ | ******....           62075000 | 1109723      4190678          3072.0     20206.7    100384419
3167.71/794.47	c ................................ | ******....           63430000 | 1137537      4280911          3168.0     20022.0    101663809
3263.94/818.53	c ................................ | ******....           64823000 | 1164359      4371886          3264.1     19859.6    103394920
3359.81/842.56	c ................................ | *******...           66204000 | 1192226      4468566          3360.0     19703.4    104194267
3455.96/866.61	c ................................ | *******...           67583000 | 1220052      4556837          3456.0     19555.1    105669864
3551.67/890.69	c ................................ | *******...           69023000 | 1244909      4641826          3552.0     19432.0    107491940
3647.71/914.78	c ................................ | *******...           70471000 | 1272942      4734693          3648.1     19317.4    108119100
3743.79/938.86	c ................................ | *******...           71812000 | 1297867      4822769          3744.0     19180.4    108615785
3839.88/962.93	c ................................ | ********..           73126000 | 1325720      4905870          3840.0     19043.2    109027958
3935.92/987.02	c ................................ | ********..           74445000 | 1354760      4987614          3936.0     18913.8    111010405
4032.04/1011.10	c ................................ | ********..           75834000 | 1383927      5068282          4032.0     18807.8    112006313
4127.76/1035.16	c ................................ | ********..           77330000 | 1412034      5156627          4128.0     18733.0    115776280
4223.85/1059.25	c ................................ | ********..           78819000 | 1438685      5240112          4224.1     18659.5    119152950
4319.94/1083.32	c ................................ | *********.           80229000 | 1467082      5322681          4320.0     18571.4    122458691
4415.64/1107.39	c ................................ | *********.           81690000 | 1495136      5402927          4416.0     18498.6    125385111
4512.12/1131.54	c ................................ | *********.           83100000 | 1521490      5486463          4512.3     18416.3    128569323
4607.82/1155.55	c ................................ | *********.           84465000 | 1547336      5565665          4608.0     18330.0    129707151
4703.89/1179.63	c ................................ | *********.           85870000 | 1575485      5645185          4704.0     18254.6    130725576
4800.01/1203.70	c CPU limit (4800) reached
4800.01/1203.70	c Time = 4800.02 seconds
4800.01/1203.70	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3259318-1304054983/watcher-3259318-1304054983 -o /tmp/evaluation-result-3259318-1304054983/solver-3259318-1304054983 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259318-1304054983.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.43 7.93 7.96 2/177 30694
/proc/meminfo: memFree=20497324/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=9760 CPUtime=0 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) R 30692 30694 30360 0 -1 4202496 499 0 0 0 0 0 0 0 22 0 1 0 595729029 9994240 441 33554432000 4194304 4338100 140733608875312 18446744073709551615 231895151301 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/30694/statm: 2440 442 147 36 0 345 0

[startup+0.02917 s]
/proc/loadavg: 7.43 7.93 7.96 2/177 30694
/proc/meminfo: memFree=20497324/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=14468 CPUtime=0.02 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) R 30692 30694 30360 0 -1 4202496 1646 0 0 0 2 0 0 0 22 0 1 0 595729029 14815232 1588 33554432000 4194304 4338100 140733608875312 18446744073709551615 231895151649 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/30694/statm: 3617 1590 148 36 0 1522 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 14468

[startup+0.101153 s]
/proc/loadavg: 7.43 7.93 7.96 2/177 30694
/proc/meminfo: memFree=20497324/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=28808 CPUtime=0.09 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) R 30692 30694 30360 0 -1 4202496 5213 0 0 0 9 0 0 0 22 0 1 0 595729029 29499392 5155 33554432000 4194304 4338100 140733608875312 18446744073709551615 231895151301 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/30694/statm: 7202 5157 148 36 0 5107 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 28808

[startup+0.301074 s]
/proc/loadavg: 7.43 7.93 7.96 2/177 30694
/proc/meminfo: memFree=20497324/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=61188 CPUtime=0.29 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) R 30692 30694 30360 0 -1 4202496 13303 0 0 0 28 1 0 0 23 0 1 0 595729029 62656512 13244 33554432000 4194304 4338100 140733608875312 18446744073709551615 231894885606 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/30694/statm: 15297 13244 153 36 0 13202 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 61188

[startup+0.700998 s]
/proc/loadavg: 7.43 7.93 7.96 2/177 30694
/proc/meminfo: memFree=20497324/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=68236 CPUtime=0.69 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) R 30692 30694 30360 0 -1 4202496 16183 0 0 0 67 2 0 0 25 0 1 0 595729029 69873664 15082 33554432000 4194304 4338100 140733608875312 18446744073709551615 4225012 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/30694/statm: 17059 15082 161 36 0 14964 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 68236

[startup+1.50079 s]
/proc/loadavg: 7.43 7.93 7.96 6/182 30699
/proc/meminfo: memFree=20332544/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=362896 CPUtime=1.78 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) S 30692 30694 30360 0 -1 4202496 46260 0 0 0 171 7 0 0 25 0 5 0 595729029 371605504 44116 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 90724 44116 176 36 0 88629 0
Current children cumulated CPU time (s) 1.78
Current children cumulated vsize (KiB) 362896

[startup+3.10049 s]
/proc/loadavg: 7.56 7.95 7.97 10/186 30703
/proc/meminfo: memFree=20203676/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=404484 CPUtime=8.18 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) S 30692 30694 30360 0 -1 4202496 57383 0 0 0 798 20 0 0 25 0 5 0 595729029 414191616 55239 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 101121 55239 176 36 0 99026 0
[pid=30694/tid=30700] ppid=30692 vsize=404484 CPUtime=1.69 cores=0,2,4,6
/proc/30694/task/30700/stat : 30700 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 4622 0 0 0 168 1 0 0 20 0 5 0 595729169 414191616 55239 33554432000 4194304 4338100 140733608875312 18446744073709551615 4284177 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30694/tid=30701] ppid=30692 vsize=404484 CPUtime=1.68 cores=0,2,4,6
/proc/30694/task/30701/stat : 30701 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 2409 0 0 0 163 5 0 0 20 0 5 0 595729169 414191616 55239 33554432000 4194304 4338100 140733608875312 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30694/tid=30702] ppid=30692 vsize=404484 CPUtime=1.68 cores=0,2,4,6
/proc/30694/task/30702/stat : 30702 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 3743 0 0 0 168 0 0 0 25 0 5 0 595729169 414191616 55239 33554432000 4194304 4338100 140733608875312 18446744073709551615 4240501 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30694/tid=30703] ppid=30692 vsize=404484 CPUtime=1.68 cores=0,2,4,6
/proc/30694/task/30703/stat : 30703 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 2309 0 0 0 162 6 0 0 20 0 5 0 595729169 414191616 55239 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 8.18
Current children cumulated vsize (KiB) 404484

[startup+6.30089 s]
/proc/loadavg: 7.56 7.95 7.97 9/186 30703
/proc/meminfo: memFree=20031320/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=473432 CPUtime=20.95 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) S 30692 30694 30360 0 -1 4202496 74254 0 0 0 2052 43 0 0 25 0 5 0 595729029 484794368 72110 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 118358 72111 176 36 0 116263 0
[pid=30694/tid=30700] ppid=30692 vsize=473432 CPUtime=4.88 cores=0,2,4,6
/proc/30694/task/30700/stat : 30700 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 10741 0 0 0 485 3 0 0 25 0 5 0 595729169 484794368 72111 33554432000 4194304 4338100 140733608875312 18446744073709551615 4256087 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30694/tid=30701] ppid=30692 vsize=473432 CPUtime=4.89 cores=0,2,4,6
/proc/30694/task/30701/stat : 30701 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 5571 0 0 0 472 17 0 0 25 0 5 0 595729169 484794368 72111 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30694/tid=30702] ppid=30692 vsize=473432 CPUtime=4.88 cores=0,2,4,6
/proc/30694/task/30702/stat : 30702 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 8458 0 0 0 487 1 0 0 25 0 5 0 595729169 484794368 72111 33554432000 4194304 4338100 140733608875312 18446744073709551615 231895186873 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30694/tid=30703] ppid=30692 vsize=473432 CPUtime=4.89 cores=0,2,4,6
/proc/30694/task/30703/stat : 30703 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 5186 0 0 0 472 17 0 0 25 0 5 0 595729169 484794368 72112 33554432000 4194304 4338100 140733608875312 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 20.95
Current children cumulated vsize (KiB) 473432

[startup+12.7007 s]
/proc/loadavg: 7.63 7.95 7.97 9/186 30703
/proc/meminfo: memFree=19721424/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=605940 CPUtime=46.52 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) S 30692 30694 30360 0 -1 4202496 106832 0 0 0 4556 96 0 0 25 0 5 0 595729029 620482560 104688 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 151485 104688 176 36 0 149390 0
[pid=30694/tid=30700] ppid=30692 vsize=605940 CPUtime=11.24 cores=0,2,4,6
/proc/30694/task/30700/stat : 30700 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 23345 0 0 0 1118 6 0 0 25 0 5 0 595729169 620482560 104688 33554432000 4194304 4338100 140733608875312 18446744073709551615 4253920 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30694/tid=30701] ppid=30692 vsize=605940 CPUtime=11.29 cores=0,2,4,6
/proc/30694/task/30701/stat : 30701 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 10753 0 0 0 1088 41 0 0 25 0 5 0 595729169 620482560 104688 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30694/tid=30702] ppid=30692 vsize=605940 CPUtime=11.27 cores=0,2,4,6
/proc/30694/task/30702/stat : 30702 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 18483 0 0 0 1125 2 0 0 25 0 5 0 595729169 620482560 104688 33554432000 4194304 4338100 140733608875312 18446744073709551615 4283712 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30694/tid=30703] ppid=30692 vsize=605940 CPUtime=11.29 cores=0,2,4,6
/proc/30694/task/30703/stat : 30703 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 9951 0 0 0 1089 40 0 0 25 0 5 0 595729169 620482560 104688 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 46.52
Current children cumulated vsize (KiB) 605940

[startup+25.5003 s]
/proc/loadavg: 7.76 7.96 7.97 9/186 30704
/proc/meminfo: memFree=19244788/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=799388 CPUtime=97.64 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) S 30692 30694 30360 0 -1 4202496 156184 0 0 0 9570 194 0 0 25 0 5 0 595729029 818573312 154040 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 199847 154040 176 36 0 197752 0
[pid=30694/tid=30700] ppid=30692 vsize=799388 CPUtime=23.96 cores=0,2,4,6
/proc/30694/task/30700/stat : 30700 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 39866 0 0 0 2384 12 0 0 25 0 5 0 595729169 818573312 154040 33554432000 4194304 4338100 140733608875312 18446744073709551615 4254418 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30694/tid=30701] ppid=30692 vsize=799388 CPUtime=24.08 cores=0,2,4,6
/proc/30694/task/30701/stat : 30701 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 19765 0 0 0 2323 85 0 0 25 0 5 0 595729169 818573312 154040 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30694/tid=30702] ppid=30692 vsize=799388 CPUtime=24.08 cores=0,2,4,6
/proc/30694/task/30702/stat : 30702 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 33871 0 0 0 2401 7 0 0 25 0 5 0 595729169 818573312 154040 33554432000 4194304 4338100 140733608875312 18446744073709551615 4284167 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30694/tid=30703] ppid=30692 vsize=799388 CPUtime=24.08 cores=0,2,4,6
/proc/30694/task/30703/stat : 30703 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 18382 0 0 0 2326 82 0 0 25 0 5 0 595729169 818573312 154040 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 97.64
Current children cumulated vsize (KiB) 799388

[startup+51.1005 s]
/proc/loadavg: 7.90 7.98 7.98 9/186 30704
/proc/meminfo: memFree=18536104/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=1126196 CPUtime=199.88 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) S 30692 30694 30360 0 -1 4202496 237156 0 0 0 19613 375 0 0 25 0 5 0 595729029 1153224704 235012 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 281549 235012 176 36 0 279454 0
[pid=30694/tid=30700] ppid=30692 vsize=1126196 CPUtime=49.52 cores=0,2,4,6
/proc/30694/task/30700/stat : 30700 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 76132 0 0 0 4929 23 0 0 25 0 5 0 595729169 1153224704 235012 33554432000 4194304 4338100 140733608875312 18446744073709551615 231894885810 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30694/tid=30701] ppid=30692 vsize=1126196 CPUtime=49.57 cores=0,2,4,6
/proc/30694/task/30701/stat : 30701 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 27482 0 0 0 4790 167 0 0 25 0 5 0 595729169 1153224704 235013 33554432000 4194304 4338100 140733608875312 18446744073709551615 231894885792 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30694/tid=30702] ppid=30692 vsize=1126196 CPUtime=49.68 cores=0,2,4,6
/proc/30694/task/30702/stat : 30702 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 60502 0 0 0 4956 12 0 0 25 0 5 0 595729169 1153224704 235013 33554432000 4194304 4338100 140733608875312 18446744073709551615 4235902 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30694/tid=30703] ppid=30692 vsize=1126196 CPUtime=49.68 cores=0,2,4,6
/proc/30694/task/30703/stat : 30703 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 28740 0 0 0 4802 166 0 0 25 0 5 0 595729169 1153224704 235013 33554432000 4194304 4338100 140733608875312 18446744073709551615 231894885818 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 199.88
Current children cumulated vsize (KiB) 1126196

[startup+102.307 s]
/proc/loadavg: 7.96 7.98 7.98 9/186 30706
/proc/meminfo: memFree=17609048/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=1496476 CPUtime=404.38 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) S 30692 30694 30360 0 -1 4202496 327636 0 0 0 39770 668 0 0 25 0 5 0 595729029 1532391424 322035 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 374119 322036 176 36 0 372024 0
[pid=30694/tid=30700] ppid=30692 vsize=1496476 CPUtime=100.7 cores=0,2,4,6
/proc/30694/task/30700/stat : 30700 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 112692 0 0 0 10035 35 0 0 25 0 5 0 595729169 1532391424 322036 33554432000 4194304 4338100 140733608875312 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30694/tid=30701] ppid=30692 vsize=1496476 CPUtime=100.49 cores=0,2,4,6
/proc/30694/task/30701/stat : 30701 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 34389 0 0 0 9748 301 0 0 25 0 5 0 595729169 1532391424 322036 33554432000 4194304 4338100 140733608875312 18446744073709551615 231894885456 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30694/tid=30702] ppid=30692 vsize=1496476 CPUtime=100.88 cores=0,2,4,6
/proc/30694/task/30702/stat : 30702 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 99495 0 0 0 10068 20 0 0 25 0 5 0 595729169 1532391424 322036 33554432000 4194304 4338100 140733608875312 18446744073709551615 4256161 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30694/tid=30703] ppid=30692 vsize=1496476 CPUtime=100.88 cores=0,2,4,6
/proc/30694/task/30703/stat : 30703 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 36761 0 0 0 9782 306 0 0 25 0 5 0 595729169 1532391424 322036 33554432000 4194304 4338100 140733608875312 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 404.38
Current children cumulated vsize (KiB) 1496476

[startup+162.301 s]
/proc/loadavg: 8.04 8.00 7.98 9/186 30707
/proc/meminfo: memFree=16854500/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=1856196 CPUtime=644.01 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) S 30692 30694 30360 0 -1 4202496 417521 0 0 0 63358 1043 0 0 25 0 5 0 595729029 1900744704 411920 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 464049 411920 176 36 0 461954 0
[pid=30694/tid=30700] ppid=30692 vsize=1856196 CPUtime=160.69 cores=0,2,4,6
/proc/30694/task/30700/stat : 30700 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 167908 0 0 0 16015 54 0 0 25 0 5 0 595729169 1900744704 411920 33554432000 4194304 4338100 140733608875312 18446744073709551615 4254041 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30694/tid=30701] ppid=30692 vsize=1856196 CPUtime=160.15 cores=0,2,4,6
/proc/30694/task/30701/stat : 30701 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 34389 0 0 0 15542 473 0 0 25 0 5 0 595729169 1900744704 411920 33554432000 4194304 4338100 140733608875312 18446744073709551615 231894885801 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30694/tid=30702] ppid=30692 vsize=1856196 CPUtime=160.87 cores=0,2,4,6
/proc/30694/task/30702/stat : 30702 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 134163 0 0 0 16059 28 0 0 25 0 5 0 595729169 1900744704 411920 33554432000 4194304 4338100 140733608875312 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30694/tid=30703] ppid=30692 vsize=1856196 CPUtime=160.87 cores=0,2,4,6
/proc/30694/task/30703/stat : 30703 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 36761 0 0 0 15605 482 0 0 25 0 5 0 595729169 1900744704 411920 33554432000 4194304 4338100 140733608875312 18446744073709551615 231894885423 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 644.01
Current children cumulated vsize (KiB) 1856196

[startup+222.301 s]
/proc/loadavg: 8.11 8.03 8.00 9/186 30709
/proc/meminfo: memFree=16202620/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=2187764 CPUtime=883.67 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) S 30692 30694 30360 0 -1 4202496 499799 0 0 0 86994 1373 0 0 25 0 5 0 595729029 2240270336 494198 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 546941 494198 176 36 0 544846 0
[pid=30694/tid=30700] ppid=30692 vsize=2187764 CPUtime=220.67 cores=0,2,4,6
/proc/30694/task/30700/stat : 30700 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 211178 0 0 0 21998 69 0 0 25 0 5 0 595729169 2240270336 494198 33554432000 4194304 4338100 140733608875312 18446744073709551615 231894886897 0 0 4096 0 0 0 0 -1 2 0 0 0

################
# More data... #
################

/proc/30694/task/30703/stat : 30703 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 60026 0 0 0 68409 1638 0 0 25 0 5 0 595729169 3840151552 849027 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2799.91
Current children cumulated vsize (KiB) 3750148

[startup+762.307 s]
/proc/loadavg: 8.19 8.11 8.03 9/186 30726
/proc/meminfo: memFree=11918468/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=3894036 CPUtime=3039.53 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) S 30692 30694 30360 0 -1 4202496 891876 0 0 0 300237 3716 0 0 25 0 5 0 595729029 3987492864 886275 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 973509 886275 176 36 0 971414 0
[pid=30694/tid=30700] ppid=30692 vsize=3894036 CPUtime=758.73 cores=0,2,4,6
/proc/30694/task/30700/stat : 30700 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 442604 0 0 0 75708 165 0 0 25 0 5 0 595729169 3987492864 886275 33554432000 4194304 4338100 140733608875312 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30694/tid=30701] ppid=30692 vsize=3894036 CPUtime=758.37 cores=0,2,4,6
/proc/30694/task/30701/stat : 30701 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 54436 0 0 0 74105 1732 0 0 25 0 5 0 595729169 3987492864 886276 33554432000 4194304 4338100 140733608875312 18446744073709551615 4239484 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30694/tid=30702] ppid=30692 vsize=3894036 CPUtime=760.55 cores=0,2,4,6
/proc/30694/task/30702/stat : 30702 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 290123 0 0 0 75992 63 0 0 25 0 5 0 595729169 3987492864 886276 33554432000 4194304 4338100 140733608875312 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30694/tid=30703] ppid=30692 vsize=3894036 CPUtime=760.45 cores=0,2,4,6
/proc/30694/task/30703/stat : 30703 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 60414 0 0 0 74296 1749 0 0 25 0 5 0 595729169 3987492864 886276 33554432000 4194304 4338100 140733608875312 18446744073709551615 4239484 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3039.53
Current children cumulated vsize (KiB) 3894036

[startup+822.3 s]
/proc/loadavg: 8.22 8.12 8.04 9/186 30728
/proc/meminfo: memFree=11528592/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=4025412 CPUtime=3279.12 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) S 30692 30694 30360 0 -1 4202496 926521 0 0 0 323953 3959 0 0 25 0 5 0 595729029 4122021888 920920 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 1006353 920920 176 36 0 1004258 0
[pid=30694/tid=30700] ppid=30692 vsize=4025412 CPUtime=818.7 cores=0,2,4,6
/proc/30694/task/30700/stat : 30700 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 459440 0 0 0 81698 172 0 0 25 0 5 0 595729169 4122021888 920920 33554432000 4194304 4338100 140733608875312 18446744073709551615 4235746 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30694/tid=30701] ppid=30692 vsize=4025412 CPUtime=818.04 cores=0,2,4,6
/proc/30694/task/30701/stat : 30701 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 55683 0 0 0 79954 1850 0 0 25 0 5 0 595729169 4122021888 920920 33554432000 4194304 4338100 140733608875312 18446744073709551615 4235746 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30694/tid=30702] ppid=30692 vsize=4025412 CPUtime=820.54 cores=0,2,4,6
/proc/30694/task/30702/stat : 30702 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 306684 0 0 0 81986 68 0 0 25 0 5 0 595729169 4122021888 920920 33554432000 4194304 4338100 140733608875312 18446744073709551615 4235770 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30694/tid=30703] ppid=30692 vsize=4025412 CPUtime=820.42 cores=0,2,4,6
/proc/30694/task/30703/stat : 30703 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 60414 0 0 0 80179 1863 0 0 25 0 5 0 595729169 4122021888 920920 33554432000 4194304 4338100 140733608875312 18446744073709551615 4235746 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3279.12
Current children cumulated vsize (KiB) 4025412

[startup+882.301 s]
/proc/loadavg: 8.13 8.11 8.03 9/186 30729
/proc/meminfo: memFree=11265680/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=4138020 CPUtime=3518.61 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) S 30692 30694 30360 0 -1 4202496 960252 0 0 0 347688 4173 0 0 25 0 5 0 595729029 4237332480 954651 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 1034505 954651 176 36 0 1032410 0
[pid=30694/tid=30700] ppid=30692 vsize=4138020 CPUtime=878.68 cores=0,2,4,6
/proc/30694/task/30700/stat : 30700 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 478267 0 0 0 87687 181 0 0 25 0 5 0 595729169 4237332480 954651 33554432000 4194304 4338100 140733608875312 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30694/tid=30701] ppid=30692 vsize=4138020 CPUtime=877.72 cores=0,2,4,6
/proc/30694/task/30701/stat : 30701 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 58514 0 0 0 85820 1952 0 0 18 0 5 0 595729169 4237332480 954651 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30694/tid=30702] ppid=30692 vsize=4138020 CPUtime=880.47 cores=0,2,4,6
/proc/30694/task/30702/stat : 30702 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 315919 0 0 0 87977 70 0 0 25 0 5 0 595729169 4237332480 954651 33554432000 4194304 4338100 140733608875312 18446744073709551615 4240510 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30694/tid=30703] ppid=30692 vsize=4138020 CPUtime=880.31 cores=0,2,4,6
/proc/30694/task/30703/stat : 30703 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 63252 0 0 0 86068 1963 0 0 25 0 5 0 595729169 4237332480 954651 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3518.61
Current children cumulated vsize (KiB) 4138020

[startup+942.301 s]
/proc/loadavg: 8.17 8.12 8.04 9/186 30731
/proc/meminfo: memFree=11147364/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=4172428 CPUtime=3757.72 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) S 30692 30694 30360 0 -1 4202496 973238 0 0 0 371540 4232 0 0 25 0 5 0 595729029 4272566272 967637 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 1043107 967637 176 36 0 1041012 0
[pid=30694/tid=30700] ppid=30692 vsize=4172428 CPUtime=938.66 cores=0,2,4,6
/proc/30694/task/30700/stat : 30700 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 487055 0 0 0 93679 187 0 0 25 0 5 0 595729169 4272566272 967637 33554432000 4194304 4338100 140733608875312 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30694/tid=30701] ppid=30692 vsize=4172428 CPUtime=937.26 cores=0,2,4,6
/proc/30694/task/30701/stat : 30701 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 60813 0 0 0 91767 1959 0 0 18 0 5 0 595729169 4272566272 967637 33554432000 4194304 4338100 140733608875312 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30694/tid=30702] ppid=30692 vsize=4172428 CPUtime=940.12 cores=0,2,4,6
/proc/30694/task/30702/stat : 30702 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 315919 0 0 0 93942 70 0 0 25 0 5 0 595729169 4272566272 967637 33554432000 4194304 4338100 140733608875312 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30694/tid=30703] ppid=30692 vsize=4172428 CPUtime=940.26 cores=0,2,4,6
/proc/30694/task/30703/stat : 30703 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 65151 0 0 0 92016 2010 0 0 25 0 5 0 595729169 4272566272 967637 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3757.72
Current children cumulated vsize (KiB) 4172428

[startup+1002.3 s]
/proc/loadavg: 8.16 8.13 8.04 9/186 30733
/proc/meminfo: memFree=10915596/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=4247500 CPUtime=3996.94 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) S 30692 30694 30360 0 -1 4202496 991964 0 0 0 395405 4289 0 0 25 0 5 0 595729029 4349440000 986363 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 1061875 986363 176 36 0 1059780 0
[pid=30694/tid=30700] ppid=30692 vsize=4247500 CPUtime=998.64 cores=0,2,4,6
/proc/30694/task/30700/stat : 30700 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 505781 0 0 0 99669 195 0 0 25 0 5 0 595729169 4349440000 986363 33554432000 4194304 4338100 140733608875312 18446744073709551615 4256749 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30694/tid=30701] ppid=30692 vsize=4247500 CPUtime=996.84 cores=0,2,4,6
/proc/30694/task/30701/stat : 30701 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 60813 0 0 0 97720 1964 0 0 18 0 5 0 595729169 4349440000 986363 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30694/tid=30702] ppid=30692 vsize=4247500 CPUtime=999.79 cores=0,2,4,6
/proc/30694/task/30702/stat : 30702 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 315919 0 0 0 99909 70 0 0 25 0 5 0 595729169 4349440000 986363 33554432000 4194304 4338100 140733608875312 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30694/tid=30703] ppid=30692 vsize=4247500 CPUtime=1000.25 cores=0,2,4,6
/proc/30694/task/30703/stat : 30703 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 65151 0 0 0 97972 2053 0 0 25 0 5 0 595729169 4349440000 986363 33554432000 4194304 4338100 140733608875312 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3996.94
Current children cumulated vsize (KiB) 4247500

[startup+1062.3 s]
/proc/loadavg: 8.11 8.12 8.04 9/186 30734
/proc/meminfo: memFree=10474628/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=4513380 CPUtime=4236.2 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) S 30692 30694 30360 0 -1 4202496 1058497 0 0 0 419263 4357 0 0 25 0 5 0 595729029 4621701120 1052896 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 1128345 1052896 176 36 0 1126250 0
[pid=30694/tid=30700] ppid=30692 vsize=4513380 CPUtime=1058.63 cores=0,2,4,6
/proc/30694/task/30700/stat : 30700 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 549133 0 0 0 105655 208 0 0 25 0 5 0 595729169 4621701120 1052896 33554432000 4194304 4338100 140733608875312 18446744073709551615 231895139244 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30694/tid=30701] ppid=30692 vsize=4513380 CPUtime=1056.42 cores=0,2,4,6
/proc/30694/task/30701/stat : 30701 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 60813 0 0 0 103672 1970 0 0 18 0 5 0 595729169 4621701120 1052896 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30694/tid=30702] ppid=30692 vsize=4513380 CPUtime=1059.46 cores=0,2,4,6
/proc/30694/task/30702/stat : 30702 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 339100 0 0 0 105871 75 0 0 25 0 5 0 595729169 4621701120 1052896 33554432000 4194304 4338100 140733608875312 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30694/tid=30703] ppid=30692 vsize=4513380 CPUtime=1060.24 cores=0,2,4,6
/proc/30694/task/30703/stat : 30703 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 65151 0 0 0 103928 2096 0 0 25 0 5 0 595729169 4621701120 1052896 33554432000 4194304 4338100 140733608875312 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4236.2
Current children cumulated vsize (KiB) 4513380

[startup+1122.3 s]
/proc/loadavg: 8.08 8.11 8.04 9/186 30736
/proc/meminfo: memFree=10086720/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=4760492 CPUtime=4475.42 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) S 30692 30694 30360 0 -1 4202496 1120636 0 0 0 443117 4425 0 0 25 0 5 0 595729029 4874743808 1115035 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 1190123 1115035 176 36 0 1188028 0
[pid=30694/tid=30700] ppid=30692 vsize=4760492 CPUtime=1118.61 cores=0,2,4,6
/proc/30694/task/30700/stat : 30700 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 592547 0 0 0 111637 224 0 0 25 0 5 0 595729169 4874743808 1115035 33554432000 4194304 4338100 140733608875312 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30694/tid=30701] ppid=30692 vsize=4760492 CPUtime=1116.01 cores=0,2,4,6
/proc/30694/task/30701/stat : 30701 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 60813 0 0 0 109625 1976 0 0 18 0 5 0 595729169 4874743808 1115035 33554432000 4194304 4338100 140733608875312 18446744073709551615 231894885752 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30694/tid=30702] ppid=30692 vsize=4760492 CPUtime=1119.13 cores=0,2,4,6
/proc/30694/task/30702/stat : 30702 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 357825 0 0 0 111834 79 0 0 25 0 5 0 595729169 4874743808 1115035 33554432000 4194304 4338100 140733608875312 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30694/tid=30703] ppid=30692 vsize=4760492 CPUtime=1120.24 cores=0,2,4,6
/proc/30694/task/30703/stat : 30703 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 65151 0 0 0 109885 2139 0 0 25 0 5 0 595729169 4874743808 1115035 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907304616 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4475.42
Current children cumulated vsize (KiB) 4760492

[startup+1182.3 s]
/proc/loadavg: 8.08 8.10 8.04 9/186 30737
/proc/meminfo: memFree=9810044/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=4863716 CPUtime=4714.67 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) S 30692 30694 30360 0 -1 4202496 1146206 0 0 0 466981 4486 0 0 25 0 5 0 595729029 4980445184 1140605 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907293157 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 1215929 1140605 176 36 0 1213834 0
[pid=30694/tid=30700] ppid=30692 vsize=4863716 CPUtime=1178.6 cores=0,2,4,6
/proc/30694/task/30700/stat : 30700 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 615663 0 0 0 117626 234 0 0 25 0 5 0 595729169 4980445184 1140605 33554432000 4194304 4338100 140733608875312 18446744073709551615 4254294 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30694/tid=30701] ppid=30692 vsize=4863716 CPUtime=1175.61 cores=0,2,4,6
/proc/30694/task/30701/stat : 30701 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 60813 0 0 0 115577 1984 0 0 18 0 5 0 595729169 4980445184 1140605 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30694/tid=30702] ppid=30692 vsize=4863716 CPUtime=1178.82 cores=0,2,4,6
/proc/30694/task/30702/stat : 30702 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 360279 0 0 0 117802 80 0 0 25 0 5 0 595729169 4980445184 1140605 33554432000 4194304 4338100 140733608875312 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30694/tid=30703] ppid=30692 vsize=4863716 CPUtime=1180.21 cores=0,2,4,6
/proc/30694/task/30703/stat : 30703 (ttsth-5-4) R 30692 30694 30360 0 -1 4202560 65151 0 0 0 115839 2182 0 0 25 0 5 0 595729169 4980445184 1140605 33554432000 4194304 4338100 140733608875312 18446744073709551615 231907316405 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4714.67
Current children cumulated vsize (KiB) 4863716



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.7 s]
/proc/loadavg: 8.05 8.09 8.04 5/180 30739
/proc/meminfo: memFree=15880864/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=0 CPUtime=4800.03 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) Z 30692 30694 30360 0 -1 4203548 1160584 0 0 0 475497 4506 0 0 18 0 2 0 595729029 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 0 0 0 0 0 0 0
[pid=30694/tid=30702] ppid=30692 vsize=0 CPUtime=1200.1 cores=0,2,4,6
/proc/30694/task/30702/stat : 30702 (ttsth-5-4) R 30692 30694 30360 0 -1 4203588 360279 0 0 0 119930 80 0 0 25 0 2 0 595729169 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.03
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.81 s]
/proc/loadavg: 8.05 8.09 8.04 5/180 30739
/proc/meminfo: memFree=15880864/32950928 swapFree=67111524/67111528
[pid=30694] ppid=30692 vsize=0 CPUtime=4800.13 cores=0,2,4,6
/proc/30694/stat : 30694 (ttsth-5-4) Z 30692 30694 30360 0 -1 4203548 1160584 0 0 0 475497 4516 0 0 18 0 2 0 595729029 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30694/statm: 0 0 0 0 0 0 0
[pid=30694/tid=30702] ppid=30692 vsize=0 CPUtime=1200.2 cores=0,2,4,6
/proc/30694/task/30702/stat : 30702 (ttsth-5-4) R 30692 30694 30360 0 -1 4203588 360279 0 0 0 119930 90 0 0 25 0 2 0 595729169 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.13
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1203.89
CPU time (s): 4800.22
CPU user time (s): 4754.97
CPU system time (s): 45.2441
CPU usage (%): 398.725
Max. virtual memory (cumulated for all children) (KiB): 4898124

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4754.97
system time used= 45.2441
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1160584
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 6944
involuntary context switches= 23250

runsolver used 1.9937 second user time and 4.5803 second system time

The end

Launcher Data 

Begin job on node114 at 2011-04-29 07:29:44
IDJOB=3259318
IDBENCH=71549
IDSOLVER=1654
FILE ID=node114/3259318-1304054983
RUNJOBID= node114-1304054983-30674
PBS_JOBID= 13170413
Free space on /tmp= 71800 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/forced/rbsat-v2400c266431gyes3.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3259318-1304054983/watcher-3259318-1304054983 -o /tmp/evaluation-result-3259318-1304054983/solver-3259318-1304054983 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259318-1304054983.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= cef3d3066fec910e94448f2404c09548
RANDOM SEED=323674612

node114.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      20497976 kB
Buffers:       2255904 kB
Cached:        9540052 kB
SwapCached:          4 kB
Active:        3413752 kB
Inactive:      8459320 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      20497976 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:            4416 kB
Writeback:           0 kB
AnonPages:       77012 kB
Mapped:          14876 kB
Slab:           516376 kB
PageTables:       4884 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   219780 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71796 MiB
End job on node114 at 2011-04-29 07:49:49