Trace number 3260814

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.11 1204.42

General information on the benchmark

Namecrafted/skvortsov/
automata-synchronization/rnd_150_29_u.cnf
MD5SUM97209c9c2612edfa7815278bcf07a2b4
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark83.7023
Satisfiable
(Un)Satisfiability was proved
Number of variables4408
Number of clauses19904
Sum of the clauses size48208
Maximum clause length3
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 211504
Number of clauses of size 38400
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.01/0.01	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260814-1304073231.cnf 
0.01/0.01	c Using SATTIMEOUT of 4800
0.01/0.01	c Using SATRAM of 15500
0.01/0.01	c Size of subset node = 32
0.01/0.01	c Variables = 4408
0.01/0.01	c Clauses = 19904
0.01/0.01	c Literals = 48208
0.01/0.01	c Reduced size:
0.01/0.01	c Variables = 4408
0.01/0.01	c Clauses = 14736
0.01/0.01	c Literals = 36376
0.01/0.01	c a2
0.01/0.02	c a1
0.29/0.39	c Independent sub-problems = 1
0.29/0.39	c Sub-problem with 3633 variables and 14736 clauses
0.29/0.39	c using permutation method 4
0.29/0.39	c count = 2
0.29/0.39	c Ordering finished 0.380 seconds
0.29/0.39	c mapped 0.380 seconds
0.29/0.39	c permuted 0.380 seconds
0.29/0.39	c about to sort
0.29/0.39	c sorted
0.39/0.45	c clauses now 14736
0.39/0.45	c init finished 0.410 seconds
0.39/0.45	c tree built 0.450 seconds
0.39/0.45	c 
0.39/0.45	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.39/0.45	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.39/0.45	c ................................ | ..........                    |                                  0.5         0.0             
0.39/0.45	c ................................ | ..........                  1 |              1                   0.5         2.2             
0.39/0.45	c ................................ | ..........                  2 |              1                   0.5         4.4             
0.39/0.45	c ................................ | ..........                  3 |              1                   0.5         6.7             
0.39/0.45	c ................................ | ..........                  4 |              1                   0.5         8.9             
0.39/0.45	c ................................ | ..........                  5 |              1                   0.5        11.1             
0.39/0.45	c ................................ | ..........               1000 |              208                 0.5      2222.2           10
95.93/24.41	c ................................ | ..........           11007000 | 70392        1500746            96.0    114656.2      1855393
191.68/48.48	c ................................ | ..........           19692000 | 138732       2800230           192.0    102562.5      3213700
287.79/72.55	c ................................ | ..........           27634000 | 204219       3989065           288.0     95951.4      4376463
383.92/96.62	c ................................ | ..........           35121000 | 266949       5154219           384.0     91460.9      5723086
479.64/120.69	c ................................ | *.........           42369000 | 327117       6266858           480.0     88268.8      6873954
575.81/144.74	c ................................ | *.........           49563000 | 386018       7364115           576.0     86046.9      7934427
671.96/168.81	c ................................ | *.........           56164000 | 442030       8499242           672.0     83576.1      8947302
767.66/192.88	c ................................ | *.........           62499000 | 496227       9659325           768.0     81377.8      9778478
863.78/216.95	c ................................ | *.........           68831000 | 550076       10744158          864.0     79665.5     10546861
959.89/241.03	c ................................ | **........           75145000 | 603314       11815443          960.0     78274.4     11395645
1055.67/265.09	c ................................ | **........           81131000 | 653770       12988180         1056.0     76827.9     12217206
1151.79/289.15	c ................................ | **........           87178000 | 703117       14174913         1152.0     75675.3     12470297
1247.88/313.23	c ................................ | **........           93369000 | 754237       15310450         1248.0     74814.9     13396330
1343.90/337.42	c ................................ | **........           99492000 | 802301       16435807         1344.0     74026.2     14497270
1439.62/361.49	c ................................ | ***.......          105434000 | 849419       17560573         1440.0     73218.1     15504133
1535.72/385.57	c ................................ | ***.......          111451000 | 898252       18687869         1536.0     72558.8     15942087
1631.85/409.64	c ................................ | ***.......          117547000 | 947478       19758248         1632.0     72025.9     16105339
1727.98/433.70	c ................................ | ***.......          123509000 | 995658       20866602         1728.0     71474.7     16492488
1823.63/457.79	c ................................ | ***.......          129706000 | 1043623      21878004         1824.0     71110.4     17178359
1919.72/481.87	c ................................ | ****......          135709000 | 1091846      22959981         1920.0     70681.8     17800540
2015.89/505.93	c ................................ | ****......          141818000 | 1139389      24027714         2016.0     70346.2     18586427
2111.62/529.99	c ................................ | ****......          147836000 | 1186230      25092568         2112.0     69998.1     19605122
2207.70/554.07	c ................................ | ****......          153753000 | 1233132      26136653         2208.0     69634.2     19922849
2303.81/578.15	c ................................ | ****......          159661000 | 1279370      27239576         2304.0     69297.3     20213865
2399.92/602.22	c ................................ | *****.....          165619000 | 1325535      28323598         2400.0     69007.6     20970680
2495.65/626.28	c ................................ | *****.....          171460000 | 1372163      29428255         2496.0     68693.9     21963454
2591.73/650.36	c ................................ | *****.....          177171000 | 1417701      30500240         2592.0     68352.7     22551884
2687.85/674.44	c ................................ | *****.....          182804000 | 1460751      31615403         2688.0     68006.9     23146073
2783.98/698.51	c ................................ | *****.....          188568000 | 1505546      32683327         2784.0     67732.5     23419913
2879.63/722.59	c ................................ | ******....          194221000 | 1549310      33768466         2880.0     67437.8     23772131
2975.68/746.67	c ................................ | ******....          199845000 | 1592631      34885952         2976.0     67152.2     24345750
3071.68/770.78	c ................................ | ******....          205426000 | 1635233      36002901         3072.0     66870.2     25229426
3167.72/794.86	c ................................ | ******....          211195000 | 1679112      37066734         3168.0     66665.1     25711257
3263.90/818.92	c ................................ | ******....          216840000 | 1723170      38144636         3264.0     66433.6     26363208
3359.94/843.01	c ................................ | *******...          222383000 | 1765650      39213955         3360.0     66185.4     27298325
3456.00/867.10	c ................................ | *******...          227856000 | 1807689      40305739         3456.0     65930.6     27802675
3551.72/891.17	c ................................ | *******...          233363000 | 1848641      41392530         3552.0     65698.9     28263215
3647.81/915.25	c ................................ | *******...          239171000 | 1893050      42386085         3648.0     65562.2     28503904
3743.93/939.32	c ................................ | *******...          244793000 | 1936902      43442435         3744.0     65382.6     28724245
3839.63/963.39	c ................................ | ********..          250420000 | 1979843      44461769         3840.0     65213.4     29103904
3936.00/987.50	c ................................ | ********..          256173000 | 2023821      45458351         3936.0     65084.4     29697959
4032.01/1011.60	c ................................ | ********..          261751000 | 2067285      46464324         4032.0     64918.2     30196870
4127.64/1035.69	c ................................ | ********..          267388000 | 2110393      47458923         4128.0     64774.1     30950305
4223.69/1059.78	c ................................ | ********..          272883000 | 2153335      48492916         4224.0     64602.8     31433479
4319.69/1083.88	c ................................ | *********.          278465000 | 2196624      49517706         4320.0     64459.3     32005089
4415.75/1107.96	c ................................ | *********.          284036000 | 2238369      50533509         4416.0     64319.7     32731121
4511.77/1132.06	c ................................ | *********.          289449000 | 2279116      51566332         4512.0     64150.9     33337461
4607.89/1156.13	c ................................ | *********.          294982000 | 2320820      52604895         4608.0     64015.1     33666587
4703.83/1180.25	c ................................ | *********.          300286000 | 2361437      53654628         4704.0     63836.2     33981008
4799.90/1204.33	c CPU limit (4800) reached
4799.90/1204.33	c Time = 4800.01 seconds
4799.90/1204.33	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3260814-1304073231/watcher-3260814-1304073231 -o /tmp/evaluation-result-3260814-1304073231/solver-3260814-1304073231 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260814-1304073231.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.18 8.07 7.73 5/181 22693
/proc/meminfo: memFree=16725116/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=9112 CPUtime=0 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) R 22691 22693 20564 0 -1 4202496 359 0 0 0 0 0 0 0 22 0 1 0 597553788 9330688 302 33554432000 4194304 4338100 140736038876832 18446744073709551615 256415052485 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/22693/statm: 2278 302 147 36 0 183 0

[startup+0.017963 s]
/proc/loadavg: 8.18 8.07 7.73 5/181 22693
/proc/meminfo: memFree=16725116/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=11172 CPUtime=0.01 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) R 22691 22693 20564 0 -1 4202496 855 0 0 0 1 0 0 0 22 0 1 0 597553788 11440128 798 33554432000 4194304 4338100 140736038876832 18446744073709551615 4221560 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/22693/statm: 2793 798 150 36 0 698 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 11172

[startup+0.100949 s]
/proc/loadavg: 8.18 8.07 7.73 5/181 22693
/proc/meminfo: memFree=16725116/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=13560 CPUtime=0.09 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) R 22691 22693 20564 0 -1 4202496 1520 0 0 0 9 0 0 0 22 0 1 0 597553788 13885440 1423 33554432000 4194304 4338100 140736038876832 18446744073709551615 4245205 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/22693/statm: 3390 1423 156 36 0 1295 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 13560

[startup+0.30091 s]
/proc/loadavg: 8.18 8.07 7.73 5/181 22693
/proc/meminfo: memFree=16725116/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=13692 CPUtime=0.29 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) R 22691 22693 20564 0 -1 4202496 1541 0 0 0 29 0 0 0 23 0 1 0 597553788 14020608 1444 33554432000 4194304 4338100 140736038876832 18446744073709551615 4297987 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/22693/statm: 3423 1444 156 36 0 1328 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 13692

[startup+0.700775 s]
/proc/loadavg: 8.18 8.07 7.73 5/181 22693
/proc/meminfo: memFree=16725116/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=307136 CPUtime=1.42 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 13190 0 0 0 137 5 0 0 25 0 5 0 597553788 314507264 13092 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 76784 13092 176 36 0 74689 0
Current children cumulated CPU time (s) 1.42
Current children cumulated vsize (KiB) 307136

[startup+1.50062 s]
/proc/loadavg: 8.18 8.07 7.73 9/186 22698
/proc/meminfo: memFree=16658868/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=308192 CPUtime=4.6 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 14241 0 0 0 442 18 0 0 25 0 5 0 597553788 315588608 14143 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 77048 14143 176 36 0 74953 0
[pid=22693/tid=22695] ppid=22691 vsize=308192 CPUtime=1.03 cores=1,3,5,7
/proc/22693/task/22695/stat : 22695 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 605 0 0 0 99 4 0 0 18 0 5 0 597553834 315588608 14143 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=22693/tid=22696] ppid=22691 vsize=308192 CPUtime=1.02 cores=1,3,5,7
/proc/22693/task/22696/stat : 22696 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 533 0 0 0 99 3 0 0 18 0 5 0 597553834 315588608 14143 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=22693/tid=22697] ppid=22691 vsize=308192 CPUtime=1.03 cores=1,3,5,7
/proc/22693/task/22697/stat : 22697 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 821 0 0 0 99 4 0 0 18 0 5 0 597553834 315588608 14143 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427204302 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=308192 CPUtime=1.03 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 483 0 0 0 99 4 0 0 18 0 5 0 597553834 315588608 14143 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4.6
Current children cumulated vsize (KiB) 308192

[startup+3.10032 s]
/proc/loadavg: 8.18 8.07 7.73 9/186 22698
/proc/meminfo: memFree=16644248/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=311848 CPUtime=10.98 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 15878 0 0 0 1049 49 0 0 25 0 5 0 597553788 319332352 15780 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 77962 15780 176 36 0 75867 0
[pid=22693/tid=22695] ppid=22691 vsize=311848 CPUtime=2.63 cores=1,3,5,7
/proc/22693/task/22695/stat : 22695 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 969 0 0 0 252 11 0 0 22 0 5 0 597553834 319332352 15780 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=22693/tid=22696] ppid=22691 vsize=311848 CPUtime=2.62 cores=1,3,5,7
/proc/22693/task/22696/stat : 22696 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 887 0 0 0 252 10 0 0 22 0 5 0 597553834 319332352 15780 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=22693/tid=22697] ppid=22691 vsize=311848 CPUtime=2.62 cores=1,3,5,7
/proc/22693/task/22697/stat : 22697 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 1453 0 0 0 249 13 0 0 19 0 5 0 597553834 319332352 15780 33554432000 4194304 4338100 140736038876832 18446744073709551615 4284130 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=311848 CPUtime=2.63 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 770 0 0 0 251 12 0 0 20 0 5 0 597553834 319332352 15780 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 10.98
Current children cumulated vsize (KiB) 311848

[startup+6.30072 s]
/proc/loadavg: 8.25 8.08 7.74 9/186 22698
/proc/meminfo: memFree=16606940/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=322420 CPUtime=23.74 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 18908 0 0 0 2250 124 0 0 25 0 5 0 597553788 330158080 18810 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 80605 18810 176 36 0 78510 0
[pid=22693/tid=22695] ppid=22691 vsize=322420 CPUtime=5.83 cores=1,3,5,7
/proc/22693/task/22695/stat : 22695 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 1640 0 0 0 554 29 0 0 25 0 5 0 597553834 330158080 18810 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=22693/tid=22696] ppid=22691 vsize=322420 CPUtime=5.82 cores=1,3,5,7
/proc/22693/task/22696/stat : 22696 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 1548 0 0 0 554 28 0 0 18 0 5 0 597553834 330158080 18810 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=22693/tid=22697] ppid=22691 vsize=322420 CPUtime=5.79 cores=1,3,5,7
/proc/22693/task/22697/stat : 22697 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 2566 0 0 0 547 32 0 0 18 0 5 0 597553834 330158080 18810 33554432000 4194304 4338100 140736038876832 18446744073709551615 4253933 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=322420 CPUtime=5.83 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 1355 0 0 0 550 33 0 0 25 0 5 0 597553834 330158080 18810 33554432000 4194304 4338100 140736038876832 18446744073709551615 4254124 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 23.74
Current children cumulated vsize (KiB) 322420

[startup+12.7005 s]
/proc/loadavg: 8.23 8.08 7.74 9/186 22698
/proc/meminfo: memFree=16525868/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=340296 CPUtime=49.26 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 24448 0 0 0 4661 265 0 0 25 0 5 0 597553788 348463104 24350 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 85074 24350 176 36 0 82979 0
[pid=22693/tid=22695] ppid=22691 vsize=340296 CPUtime=12.21 cores=1,3,5,7
/proc/22693/task/22695/stat : 22695 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 2954 0 0 0 1158 63 0 0 23 0 5 0 597553834 348463104 24350 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=22693/tid=22696] ppid=22691 vsize=340296 CPUtime=12.21 cores=1,3,5,7
/proc/22693/task/22696/stat : 22696 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 2764 0 0 0 1157 64 0 0 22 0 5 0 597553834 348463104 24350 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=22693/tid=22697] ppid=22691 vsize=340296 CPUtime=12.15 cores=1,3,5,7
/proc/22693/task/22697/stat : 22697 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 4444 0 0 0 1147 68 0 0 23 0 5 0 597553834 348463104 24350 33554432000 4194304 4338100 140736038876832 18446744073709551615 4239571 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=340296 CPUtime=12.21 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 2487 0 0 0 1154 67 0 0 20 0 5 0 597553834 348463104 24350 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 49.26
Current children cumulated vsize (KiB) 340296

[startup+25.5011 s]
/proc/loadavg: 8.25 8.09 7.75 9/186 22698
/proc/meminfo: memFree=16402984/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=365108 CPUtime=100.33 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 34033 0 0 0 9482 551 0 0 25 0 5 0 597553788 373870592 33935 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 91277 33935 176 36 0 89182 0
[pid=22693/tid=22695] ppid=22691 vsize=365108 CPUtime=24.92 cores=1,3,5,7
/proc/22693/task/22695/stat : 22695 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 5592 0 0 0 2359 133 0 0 25 0 5 0 597553834 373870592 33935 33554432000 4194304 4338100 140736038876832 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=22693/tid=22696] ppid=22691 vsize=365108 CPUtime=24.99 cores=1,3,5,7
/proc/22693/task/22696/stat : 22696 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 4802 0 0 0 2366 133 0 0 25 0 5 0 597553834 373870592 33935 33554432000 4194304 4338100 140736038876832 18446744073709551615 256414786809 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=22693/tid=22697] ppid=22691 vsize=365108 CPUtime=24.93 cores=1,3,5,7
/proc/22693/task/22697/stat : 22697 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 7198 0 0 0 2351 142 0 0 20 0 5 0 597553834 373870592 33935 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=365108 CPUtime=25 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 4642 0 0 0 2359 141 0 0 25 0 5 0 597553834 373870592 33935 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 100.33
Current children cumulated vsize (KiB) 365108

[startup+51.1003 s]
/proc/loadavg: 8.16 8.08 7.75 9/186 22699
/proc/meminfo: memFree=16232356/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=422068 CPUtime=202.43 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 50514 0 0 0 19135 1108 0 0 25 0 5 0 597553788 432197632 50416 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 105517 50416 176 36 0 103422 0
[pid=22693/tid=22695] ppid=22691 vsize=422068 CPUtime=50.34 cores=1,3,5,7
/proc/22693/task/22695/stat : 22695 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 10587 0 0 0 4757 277 0 0 25 0 5 0 597553834 432197632 50416 33554432000 4194304 4338100 140736038876832 18446744073709551615 4285817 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=22693/tid=22696] ppid=22691 vsize=422068 CPUtime=50.55 cores=1,3,5,7
/proc/22693/task/22696/stat : 22696 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 8738 0 0 0 4781 274 0 0 25 0 5 0 597553834 432197632 50416 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=22693/tid=22697] ppid=22691 vsize=422068 CPUtime=50.5 cores=1,3,5,7
/proc/22693/task/22697/stat : 22697 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 11238 0 0 0 4775 275 0 0 25 0 5 0 597553834 432197632 50416 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=422068 CPUtime=50.56 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 8152 0 0 0 4776 280 0 0 25 0 5 0 597553834 432197632 50416 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 202.43
Current children cumulated vsize (KiB) 422068

[startup+102.307 s]
/proc/loadavg: 8.07 8.07 7.76 9/186 22700
/proc/meminfo: memFree=15912548/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=513840 CPUtime=406.68 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 80498 0 0 0 38390 2278 0 0 25 0 5 0 597553788 526172160 80400 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 128460 80400 176 36 0 126365 0
[pid=22693/tid=22695] ppid=22691 vsize=513840 CPUtime=101.37 cores=1,3,5,7
/proc/22693/task/22695/stat : 22695 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 19845 0 0 0 9566 571 0 0 18 0 5 0 597553834 526172160 80400 33554432000 4194304 4338100 140736038876832 18446744073709551615 256414786976 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=22693/tid=22696] ppid=22691 vsize=513840 CPUtime=101.59 cores=1,3,5,7
/proc/22693/task/22696/stat : 22696 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 15858 0 0 0 9598 561 0 0 25 0 5 0 597553834 526172160 80400 33554432000 4194304 4338100 140736038876832 18446744073709551615 256414788081 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=22693/tid=22697] ppid=22691 vsize=513840 CPUtime=101.62 cores=1,3,5,7
/proc/22693/task/22697/stat : 22697 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 19018 0 0 0 9592 570 0 0 18 0 5 0 597553834 526172160 80400 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=513840 CPUtime=101.61 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 13978 0 0 0 9588 573 0 0 22 0 5 0 597553834 526172160 80400 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 406.68
Current children cumulated vsize (KiB) 513840

[startup+162.3 s]
/proc/loadavg: 8.11 8.08 7.79 9/186 22702
/proc/meminfo: memFree=15531620/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=608908 CPUtime=646.02 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 111093 0 0 0 61154 3448 0 0 25 0 5 0 597553788 623521792 110995 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 152227 110995 176 36 0 150132 0
[pid=22693/tid=22695] ppid=22691 vsize=608908 CPUtime=160.94 cores=1,3,5,7
/proc/22693/task/22695/stat : 22695 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 30356 0 0 0 15225 869 0 0 25 0 5 0 597553834 623521792 110995 33554432000 4194304 4338100 140736038876832 18446744073709551615 256415389755 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=22693/tid=22696] ppid=22691 vsize=608908 CPUtime=161.49 cores=1,3,5,7
/proc/22693/task/22696/stat : 22696 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 24528 0 0 0 15306 843 0 0 25 0 5 0 597553834 623521792 110995 33554432000 4194304 4338100 140736038876832 18446744073709551615 4253954 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=22693/tid=22697] ppid=22691 vsize=608908 CPUtime=161.55 cores=1,3,5,7
/proc/22693/task/22697/stat : 22697 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 26365 0 0 0 15292 863 0 0 25 0 5 0 597553834 623521792 110995 33554432000 4194304 4338100 140736038876832 18446744073709551615 4286738 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=608908 CPUtime=161.55 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 18045 0 0 0 15285 870 0 0 25 0 5 0 597553834 623521792 110995 33554432000 4194304 4338100 140736038876832 18446744073709551615 4241570 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 646.02
Current children cumulated vsize (KiB) 608908


################
# More data... #
################

/proc/meminfo: memFree=12825268/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=1373580 CPUtime=2799.1 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 317489 0 0 0 266306 13604 0 0 25 0 5 0 597553788 1406545920 317391 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 343395 317391 176 36 0 341300 0
[pid=22693/tid=22695] ppid=22691 vsize=1373580 CPUtime=698.48 cores=1,3,5,7
/proc/22693/task/22695/stat : 22695 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 112452 0 0 0 66366 3482 0 0 19 0 5 0 597553834 1406545920 317391 33554432000 4194304 4338100 140736038876832 18446744073709551615 4254041 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=22693/tid=22696] ppid=22691 vsize=1373580 CPUtime=699.47 cores=1,3,5,7
/proc/22693/task/22696/stat : 22696 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 85335 0 0 0 66608 3339 0 0 20 0 5 0 597553834 1406545920 317391 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=22693/tid=22697] ppid=22691 vsize=1373580 CPUtime=700.4 cores=1,3,5,7
/proc/22693/task/22697/stat : 22697 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 64195 0 0 0 66689 3351 0 0 20 0 5 0 597553834 1406545920 317391 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=1373580 CPUtime=700.27 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 43708 0 0 0 66598 3429 0 0 25 0 5 0 597553834 1406545920 317391 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2799.1
Current children cumulated vsize (KiB) 1373580

[startup+762.306 s]
/proc/loadavg: 8.07 8.09 7.92 9/186 22721
/proc/meminfo: memFree=12603180/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=1435520 CPUtime=3038.22 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 339189 0 0 0 289255 14567 0 0 25 0 5 0 597553788 1469972480 339091 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 358880 339091 176 36 0 356785 0
[pid=22693/tid=22695] ppid=22691 vsize=1435520 CPUtime=758.15 cores=1,3,5,7
/proc/22693/task/22695/stat : 22695 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 121329 0 0 0 72084 3731 0 0 25 0 5 0 597553834 1469972480 339091 33554432000 4194304 4338100 140736038876832 18446744073709551615 256414788081 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=22693/tid=22696] ppid=22691 vsize=1435520 CPUtime=759.29 cores=1,3,5,7
/proc/22693/task/22696/stat : 22696 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 92864 0 0 0 72355 3574 0 0 25 0 5 0 597553834 1469972480 339091 33554432000 4194304 4338100 140736038876832 18446744073709551615 256414786829 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=22693/tid=22697] ppid=22691 vsize=1435520 CPUtime=760.31 cores=1,3,5,7
/proc/22693/task/22697/stat : 22697 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 67051 0 0 0 72443 3588 0 0 25 0 5 0 597553834 1469972480 339091 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=1435520 CPUtime=759.98 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 46146 0 0 0 72327 3671 0 0 25 0 5 0 597553834 1469972480 339091 33554432000 4194304 4338100 140736038876832 18446744073709551615 4243536 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3038.22
Current children cumulated vsize (KiB) 1435520

[startup+822.301 s]
/proc/loadavg: 8.02 8.07 7.92 9/186 22723
/proc/meminfo: memFree=12353680/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=1689080 CPUtime=3277.44 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 361527 0 0 0 312351 15393 0 0 25 0 5 0 597553788 1729617920 361429 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 422270 361429 176 36 0 420175 0
[pid=22693/tid=22695] ppid=22691 vsize=1689080 CPUtime=817.9 cores=1,3,5,7
/proc/22693/task/22695/stat : 22695 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 131086 0 0 0 77843 3947 0 0 19 0 5 0 597553834 1729617920 361429 33554432000 4194304 4338100 140736038876832 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=22693/tid=22696] ppid=22691 vsize=1689080 CPUtime=819.15 cores=1,3,5,7
/proc/22693/task/22696/stat : 22696 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 100201 0 0 0 78144 3771 0 0 25 0 5 0 597553834 1729617920 361429 33554432000 4194304 4338100 140736038876832 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=22693/tid=22697] ppid=22691 vsize=1689080 CPUtime=820.16 cores=1,3,5,7
/proc/22693/task/22697/stat : 22697 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 69829 0 0 0 78218 3798 0 0 25 0 5 0 597553834 1729617920 361429 33554432000 4194304 4338100 140736038876832 18446744073709551615 256414786594 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=1689080 CPUtime=819.76 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 48612 0 0 0 78101 3875 0 0 25 0 5 0 597553834 1729617920 361429 33554432000 4194304 4338100 140736038876832 18446744073709551615 256414786669 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3277.44
Current children cumulated vsize (KiB) 1689080

[startup+882.301 s]
/proc/loadavg: 8.06 8.07 7.93 9/186 22724
/proc/meminfo: memFree=12126868/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=1745768 CPUtime=3516.64 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 382923 0 0 0 335275 16389 0 0 25 0 5 0 597553788 1787666432 382825 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 436442 382825 176 36 0 434347 0
[pid=22693/tid=22695] ppid=22691 vsize=1745768 CPUtime=877.51 cores=1,3,5,7
/proc/22693/task/22695/stat : 22695 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 140532 0 0 0 83546 4205 0 0 21 0 5 0 597553834 1787666432 382825 33554432000 4194304 4338100 140736038876832 18446744073709551615 256414787041 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=22693/tid=22696] ppid=22691 vsize=1745768 CPUtime=878.99 cores=1,3,5,7
/proc/22693/task/22696/stat : 22696 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 107280 0 0 0 83888 4011 0 0 21 0 5 0 597553834 1787666432 382825 33554432000 4194304 4338100 140736038876832 18446744073709551615 256414786683 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=22693/tid=22697] ppid=22691 vsize=1745768 CPUtime=879.98 cores=1,3,5,7
/proc/22693/task/22697/stat : 22697 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 72255 0 0 0 83953 4045 0 0 25 0 5 0 597553834 1787666432 382825 33554432000 4194304 4338100 140736038876832 18446744073709551615 256414786941 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=1745768 CPUtime=879.68 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 51057 0 0 0 83843 4125 0 0 25 0 5 0 597553834 1787666432 382825 33554432000 4194304 4338100 140736038876832 18446744073709551615 4206902 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3516.64
Current children cumulated vsize (KiB) 1745768

[startup+942.3 s]
/proc/loadavg: 8.16 8.10 7.94 9/186 22726
/proc/meminfo: memFree=11847352/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=1782592 CPUtime=3755.86 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 398954 0 0 0 358044 17542 0 0 25 0 5 0 597553788 1825374208 398856 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 445648 398856 176 36 0 443553 0
[pid=22693/tid=22695] ppid=22691 vsize=1782592 CPUtime=937.37 cores=1,3,5,7
/proc/22693/task/22695/stat : 22695 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 147290 0 0 0 89238 4499 0 0 18 0 5 0 597553834 1825374208 398856 33554432000 4194304 4338100 140736038876832 18446744073709551615 256414787041 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=22693/tid=22696] ppid=22691 vsize=1782592 CPUtime=938.77 cores=1,3,5,7
/proc/22693/task/22696/stat : 22696 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 111635 0 0 0 89571 4306 0 0 25 0 5 0 597553834 1825374208 398856 33554432000 4194304 4338100 140736038876832 18446744073709551615 256415088256 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=22693/tid=22697] ppid=22691 vsize=1782592 CPUtime=939.75 cores=1,3,5,7
/proc/22693/task/22697/stat : 22697 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 74529 0 0 0 89650 4325 0 0 25 0 5 0 597553834 1825374208 398856 33554432000 4194304 4338100 140736038876832 18446744073709551615 4285675 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=1782592 CPUtime=939.5 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 53701 0 0 0 89540 4410 0 0 25 0 5 0 597553834 1825374208 398856 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3755.86
Current children cumulated vsize (KiB) 1782592

[startup+1002.3 s]
/proc/loadavg: 8.16 8.11 7.95 8/186 22727
/proc/meminfo: memFree=11627988/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=1829104 CPUtime=3994.93 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 416589 0 0 0 380795 18698 0 0 25 0 5 0 597553788 1873002496 416491 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 457276 416491 176 36 0 455181 0
[pid=22693/tid=22695] ppid=22691 vsize=1829104 CPUtime=997.16 cores=1,3,5,7
/proc/22693/task/22695/stat : 22695 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 155662 0 0 0 94920 4796 0 0 25 0 5 0 597553834 1873002496 416491 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217268 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=22693/tid=22696] ppid=22691 vsize=1829104 CPUtime=998.39 cores=1,3,5,7
/proc/22693/task/22696/stat : 22696 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 116378 0 0 0 95247 4592 0 0 25 0 5 0 597553834 1873002496 416491 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=22693/tid=22697] ppid=22691 vsize=1829104 CPUtime=999.64 cores=1,3,5,7
/proc/22693/task/22697/stat : 22697 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 76791 0 0 0 95362 4602 0 0 18 0 5 0 597553834 1873002496 416491 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=1829104 CPUtime=999.24 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 55959 0 0 0 95219 4705 0 0 25 0 5 0 597553834 1873002496 416491 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3994.93
Current children cumulated vsize (KiB) 1829104

[startup+1062.3 s]
/proc/loadavg: 8.18 8.12 7.96 9/186 22761
/proc/meminfo: memFree=16659240/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=1885000 CPUtime=4234.05 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 438152 0 0 0 403796 19609 0 0 25 0 5 0 597553788 1930240000 438054 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 471250 438054 176 36 0 469155 0
[pid=22693/tid=22695] ppid=22691 vsize=1885000 CPUtime=1056.66 cores=1,3,5,7
/proc/22693/task/22695/stat : 22695 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 166137 0 0 0 100626 5040 0 0 25 0 5 0 597553834 1930240000 438054 33554432000 4194304 4338100 140736038876832 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=22693/tid=22696] ppid=22691 vsize=1885000 CPUtime=1058.22 cores=1,3,5,7
/proc/22693/task/22696/stat : 22696 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 122780 0 0 0 101007 4815 0 0 25 0 5 0 597553834 1930240000 438054 33554432000 4194304 4338100 140736038876832 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=22693/tid=22697] ppid=22691 vsize=1885000 CPUtime=1059.53 cores=1,3,5,7
/proc/22693/task/22697/stat : 22697 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 79139 0 0 0 101133 4820 0 0 25 0 5 0 597553834 1930240000 438054 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=1885000 CPUtime=1059.16 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 58297 0 0 0 100984 4932 0 0 23 0 5 0 597553834 1930240000 438054 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4234.05
Current children cumulated vsize (KiB) 1885000

[startup+1122.3 s]
/proc/loadavg: 8.11 8.11 7.97 9/186 22763
/proc/meminfo: memFree=16519944/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=1944156 CPUtime=4473.11 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 459768 0 0 0 426702 20609 0 0 25 0 5 0 597553788 1990815744 459670 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 486039 459670 176 36 0 483944 0
[pid=22693/tid=22695] ppid=22691 vsize=1944156 CPUtime=1116.35 cores=1,3,5,7
/proc/22693/task/22695/stat : 22695 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 176124 0 0 0 106336 5299 0 0 25 0 5 0 597553834 1990815744 459670 33554432000 4194304 4338100 140736038876832 18446744073709551615 4254433 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=22693/tid=22696] ppid=22691 vsize=1944156 CPUtime=1117.88 cores=1,3,5,7
/proc/22693/task/22696/stat : 22696 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 129571 0 0 0 106727 5061 0 0 25 0 5 0 597553834 1990815744 459670 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=22693/tid=22697] ppid=22691 vsize=1944156 CPUtime=1119.35 cores=1,3,5,7
/proc/22693/task/22697/stat : 22697 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 81455 0 0 0 106872 5063 0 0 22 0 5 0 597553834 1990815744 459670 33554432000 4194304 4338100 140736038876832 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=1944156 CPUtime=1119.04 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 60819 0 0 0 106721 5183 0 0 25 0 5 0 597553834 1990815744 459670 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4473.11
Current children cumulated vsize (KiB) 1944156

[startup+1182.3 s]
/proc/loadavg: 8.04 8.09 7.97 9/186 22764
/proc/meminfo: memFree=16398164/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=2005288 CPUtime=4712.2 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) S 22691 22693 20564 0 -1 4202496 478333 0 0 0 449602 21618 0 0 25 0 5 0 597553788 2053414912 478235 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 501322 478235 176 36 0 499227 0
[pid=22693/tid=22695] ppid=22691 vsize=2005288 CPUtime=1176.17 cores=1,3,5,7
/proc/22693/task/22695/stat : 22695 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 183310 0 0 0 112055 5562 0 0 25 0 5 0 597553834 2053414912 478235 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=22693/tid=22696] ppid=22691 vsize=2005288 CPUtime=1177.69 cores=1,3,5,7
/proc/22693/task/22696/stat : 22696 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 135007 0 0 0 112462 5307 0 0 25 0 5 0 597553834 2053414912 478235 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=22693/tid=22697] ppid=22691 vsize=2005288 CPUtime=1179.04 cores=1,3,5,7
/proc/22693/task/22697/stat : 22697 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 84745 0 0 0 112595 5309 0 0 25 0 5 0 597553834 2053414912 478235 33554432000 4194304 4338100 140736038876832 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=2005288 CPUtime=1178.81 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4202560 63472 0 0 0 112444 5437 0 0 18 0 5 0 597553834 2053414912 478235 33554432000 4194304 4338100 140736038876832 18446744073709551615 4254239 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4712.2
Current children cumulated vsize (KiB) 2005288



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1204.4 s]
/proc/loadavg: 8.02 8.08 7.97 6/183 22765
/proc/meminfo: memFree=17975960/32950928 swapFree=67111524/67111528
[pid=22693] ppid=22691 vsize=0 CPUtime=4800.09 cores=1,3,5,7
/proc/22693/stat : 22693 (ttsth-5-4) Z 22691 22693 20564 0 -1 4203548 485809 0 0 0 457977 22032 0 0 18 0 2 0 597553788 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22693/statm: 0 0 0 0 0 0 0
[pid=22693/tid=22698] ppid=22691 vsize=0 CPUtime=1200.77 cores=1,3,5,7
/proc/22693/task/22698/stat : 22698 (ttsth-5-4) R 22691 22693 20564 0 -1 4203588 64349 0 0 0 114531 5546 0 0 25 0 2 0 597553834 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.09
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 0
Real time (s): 1204.42
CPU time (s): 4800.11
CPU user time (s): 4579.77
CPU system time (s): 220.333
CPU usage (%): 398.541
Max. virtual memory (cumulated for all children) (KiB): 2024492

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4579.77
system time used= 220.334
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 485809
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1127329
involuntary context switches= 30474

runsolver used 2.09368 second user time and 4.48832 second system time

The end

Launcher Data 

Begin job on node115 at 2011-04-29 12:33:51
IDJOB=3260814
IDBENCH=83196
IDSOLVER=1654
FILE ID=node115/3260814-1304073231
RUNJOBID= node115-1304071862-22508
PBS_JOBID= 13170430
Free space on /tmp= 72732 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/skvortsov/automata-synchronization/rnd_150_29_u.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3260814-1304073231/watcher-3260814-1304073231 -o /tmp/evaluation-result-3260814-1304073231/solver-3260814-1304073231 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260814-1304073231.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 97209c9c2612edfa7815278bcf07a2b4
RANDOM SEED=1147981685

node115.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.62
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      16725396 kB
Buffers:       2320260 kB
Cached:       11541704 kB
SwapCached:          4 kB
Active:        8582504 kB
Inactive:      7086056 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      16725396 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:             428 kB
Writeback:           0 kB
AnonPages:     1805948 kB
Mapped:          15336 kB
Slab:           489908 kB
PageTables:       7768 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:  1999088 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72732 MiB
End job on node115 at 2011-04-29 12:53:58