Trace number 3266930

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.44 1206.23

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m27-q255-n25-p15-s2076598.cnf
MD5SUMf5a22d1154b549fd98215f1710a15f46
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.357944
Satisfiable
(Un)Satisfiability was proved
Number of variables545
Number of clauses29734
Sum of the clauses size59963
Maximum clause length27
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 229709
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 525

Solver Data 

0.00/0.01	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3266930-1304107450.cnf 
0.00/0.01	c Using SATTIMEOUT of 4800
0.00/0.01	c Using SATRAM of 15500
0.00/0.01	c Size of subset node = 32
0.00/0.01	c Variables = 545
0.00/0.01	c Clauses = 29734
0.00/0.01	c Literals = 59963
0.00/0.01	c Reduced size:
0.00/0.01	c Variables = 545
0.00/0.01	c Clauses = 29734
0.00/0.01	c Literals = 59963
0.00/0.01	c a2
0.00/0.03	c a1
0.03/0.06	c Independent sub-problems = 1
0.03/0.06	c Sub-problem with 545 variables and 29734 clauses
0.03/0.06	c using permutation method 4
0.03/0.06	c count = 1
0.03/0.06	c Ordering finished 0.050 seconds
0.03/0.06	c mapped 0.060 seconds
0.03/0.06	c permuted 0.060 seconds
0.03/0.06	c about to sort
0.03/0.06	c sorted
0.09/0.12	c clauses now 29734
0.09/0.12	c init finished 0.070 seconds
0.09/0.12	c tree built 0.120 seconds
0.09/0.12	c 
0.09/0.12	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.09/0.12	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.09/0.12	c ................................ | ..........                    |                                  0.1         0.0             
0.09/0.12	c ................................ | ..........                  1 |              27                  0.1         8.3           11
0.09/0.12	c ................................ | ..........                  2 |              29                  0.1        16.7           11
0.09/0.12	c ................................ | ..........                  3 |              30                  0.1        25.0           11
0.09/0.12	c ................................ | ..........                  4 |              30                  0.1        33.3           11
0.09/0.12	c ................................ | ..........                  5 |              30                  0.1        41.7           11
0.09/0.13	c ................................ | ..........               1000 | 2            282                 0.1      7692.3         7398
95.82/24.24	c ................................ | ..........            5506000 | 36523        1187648            96.0     57348.2     24151280
191.85/48.33	c ................................ | ..........           10358000 | 79372        2261215           192.0     53947.9     41959189
287.61/72.39	c ................................ | ..........           14781000 | 129961       3150520           288.0     51321.1     56708662
383.78/96.46	c ................................ | ..........           18635000 | 172318       3879165           384.0     48527.4     69437381
479.95/120.52	c ................................ | *.........           22710000 | 206422       4631880           480.0     47310.5     84309654
575.67/144.58	c ................................ | *.........           26181000 | 239523       5253217           576.0     45453.1     92174666
671.72/168.67	c ................................ | *.........           30071000 | 276248       5942345           672.0     44748.5    100780912
767.81/192.74	c ................................ | *.........           33721000 | 318496       6516656           768.0     43907.6    108366220
863.72/216.87	c ................................ | *.........           37053000 | 354057       7130556           864.0     42884.4    119153771
959.98/241.00	c ................................ | **........           40347000 | 393290       7683737           960.0     42028.1    128950855
1055.82/265.15	c ................................ | **........           43491000 | 433068       8241684          1056.0     41183.9    135410991
1151.62/289.29	c ................................ | **........           46581000 | 471411       8724427          1152.0     40434.9    144405485
1247.90/313.42	c ................................ | **........           49274000 | 508393       9223956          1248.0     39482.4    152094567
1343.87/337.53	c ................................ | **........           52494000 | 552284       9722114          1344.0     39058.0    159455292
1439.66/361.69	c ................................ | ***.......           55354000 | 593095       10220436         1440.0     38439.5    160267187
1535.91/385.82	c ................................ | ***.......           58241000 | 631266       10807350         1536.0     37917.1    165620150
1631.77/409.96	c ................................ | ***.......           61172000 | 675404       11371101         1632.0     37482.8    174577772
1727.65/434.09	c ................................ | ***.......           63870000 | 712339       11855860         1728.0     36961.8    183073596
1823.93/458.22	c ................................ | ***.......           67058000 | 741342       12457064         1824.0     36764.3    192634761
1919.73/482.37	c ................................ | ****......           69780000 | 764590       12954197         1920.0     36343.6    197657211
2015.71/506.47	c ................................ | ****......           72439000 | 792358       13424798         2016.0     35931.7    202927396
2111.88/530.53	c ................................ | ****......           74946000 | 822746       13847941         2112.0     35485.8    211215054
2207.87/554.64	c ................................ | ****......           77472000 | 853234       14461172         2208.1     35086.2    218253196
2303.90/578.73	c ................................ | ****......           79994000 | 878502       15066766         2304.0     34719.2    221936554
2399.96/602.81	c ................................ | *****.....           82499000 | 908459       15651527         2400.0     34374.2    227300223
2495.63/626.89	c ................................ | *****.....           85040000 | 938654       16225108         2496.0     34070.5    236608641
2591.72/650.97	c ................................ | *****.....           87439000 | 969321       16736233         2592.0     33734.2    244996656
2687.79/675.06	c ................................ | *****.....           89683000 | 999733       17303764         2688.0     33364.1    255569000
2784.53/699.31	c ................................ | *****.....           91840000 | 1026659      17816816         2784.6     32981.8    255569000
2879.76/723.26	c ................................ | ******....           94161000 | 1044850      18377068         2880.0     32694.7    257700981
2975.84/747.34	c ................................ | ******....           96437000 | 1065507      18876148         2976.0     32404.9    263212632
3071.92/771.43	c ................................ | ******....           98828000 | 1089765      19401398         3072.0     32170.3    270363190
3167.90/795.53	c ................................ | ******....          100986000 | 1121499      19884305         3168.0     31876.7    277475164
3263.78/819.67	c ................................ | ******....          103061000 | 1155093      20331641         3264.1     31574.6    285349252
3359.93/843.72	c ................................ | *******...          105116000 | 1189080      20763517         3360.0     31284.3    290937738
3455.95/867.81	c ................................ | *******...          108304000 | 1217623      21319212         3456.0     31338.0    297075886
3551.75/891.96	c ................................ | *******...          111350000 | 1245521      21817588         3552.0     31348.4    303385794
3647.95/916.11	c ................................ | *******...          114200000 | 1272354      22331179         3648.0     31304.7    312179832
3743.70/940.28	c ................................ | *******...          117041000 | 1297038      22868464         3744.0     31260.8    314426817
3839.77/964.46	c ................................ | ********..          119662000 | 1324197      23378383         3840.0     31161.9    318824532
3935.97/988.61	c ................................ | ********..          122080000 | 1345663      23820940         3936.0     31016.1    320797635
4032.03/1012.80	c ................................ | ********..          124716000 | 1367105      24413283         4032.0     30931.3    321928411
4127.93/1036.92	c ................................ | ********..          127258000 | 1393968      24962040         4128.0     30827.9    325384260
4223.75/1061.06	c ................................ | ********..          129607000 | 1419092      25527118         4224.0     30683.5    326229736
4319.68/1085.18	c ................................ | *********.          132247000 | 1446110      26137101         4320.0     30612.7    332729446
4415.98/1109.31	c ................................ | *********.          134866000 | 1469818      26691899         4416.0     30540.1    335527012
4511.91/1133.42	c ................................ | *********.          137335000 | 1498476      27226710         4512.0     30437.7    335828245
4607.84/1157.54	c ................................ | *********.          139708000 | 1522368      27698101         4608.0     30318.4    337423981
4703.99/1181.70	c ................................ | *********.          142228000 | 1551048      28195904         4704.0     30235.5    342712558
4799.62/1205.79	c CPU limit (4800) reached
4799.62/1205.79	c Time = 4800.00 seconds
4799.62/1205.79	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3266930-1304107450/watcher-3266930-1304107450 -o /tmp/evaluation-result-3266930-1304107450/solver-3266930-1304107450 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3266930-1304107450.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.26 7.89 7.94 2/177 17735
/proc/meminfo: memFree=30577196/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=9756 CPUtime=0 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) R 17733 17735 17262 0 -1 4202496 492 0 0 0 0 0 0 0 25 0 1 0 522343660 9990144 434 33554432000 4194304 4338100 140736114325168 18446744073709551615 251813900997 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17735/statm: 2439 435 147 36 0 344 0

[startup+0.0416771 s]
/proc/loadavg: 7.26 7.89 7.94 2/177 17735
/proc/meminfo: memFree=30577196/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=13416 CPUtime=0.03 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) R 17733 17735 17262 0 -1 4202496 1952 0 0 0 3 0 0 0 25 0 1 0 522343660 13737984 1285 33554432000 4194304 4338100 140736114325168 18446744073709551615 251813900997 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17735/statm: 3354 1287 155 36 0 1259 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 13416

[startup+0.100757 s]
/proc/loadavg: 7.26 7.89 7.94 2/177 17735
/proc/meminfo: memFree=30577196/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=24328 CPUtime=0.09 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) R 17733 17735 17262 0 -1 4202496 4829 0 0 0 9 0 0 0 25 0 1 0 522343660 24911872 4103 33554432000 4194304 4338100 140736114325168 18446744073709551615 4278345 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17735/statm: 6082 4111 164 36 0 3987 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 24328

[startup+0.449556 s]
/proc/loadavg: 7.26 7.89 7.94 2/177 17735
/proc/meminfo: memFree=30577196/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=224068 CPUtime=1.4 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 10703 0 0 0 138 2 0 0 25 0 5 0 522343660 229445632 9977 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 59960 12916 176 36 0 57865 0
Current children cumulated CPU time (s) 1.4
Current children cumulated vsize (KiB) 224068

[startup+1.0705 s]
/proc/loadavg: 7.26 7.89 7.94 2/177 17735
/proc/meminfo: memFree=30577196/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=255744 CPUtime=3.87 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 18472 0 0 0 384 3 0 0 25 0 5 0 522343660 261881856 17746 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 63936 17746 176 36 0 61841 0
Current children cumulated CPU time (s) 3.87
Current children cumulated vsize (KiB) 255744

[startup+1.55576 s]
/proc/loadavg: 7.26 7.89 7.94 2/177 17735
/proc/meminfo: memFree=30577196/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=278320 CPUtime=5.81 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 24148 0 0 0 576 5 0 0 25 0 5 0 522343660 284999680 23422 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 69580 23422 176 36 0 67485 0
Current children cumulated CPU time (s) 5.81
Current children cumulated vsize (KiB) 278320

[startup+3.10111 s]
/proc/loadavg: 7.26 7.89 7.94 9/186 17744
/proc/meminfo: memFree=30256764/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=340928 CPUtime=11.98 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 40854 0 0 0 1188 10 0 0 25 0 5 0 522343660 349110272 40128 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 85232 40128 176 36 0 83137 0
[pid=17735/tid=17737] ppid=17733 vsize=340928 CPUtime=2.94 cores=0,2,4,6
/proc/17735/task/17737/stat : 17737 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 11997 0 0 0 291 3 0 0 20 0 5 0 522343672 349110272 40129 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826052892 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=17735/tid=17738] ppid=17733 vsize=340928 CPUtime=2.95 cores=0,2,4,6
/proc/17735/task/17738/stat : 17738 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 7393 0 0 0 294 1 0 0 19 0 5 0 522343672 349110272 40129 33554432000 4194304 4338100 140736114325168 18446744073709551615 4206902 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=17735/tid=17739] ppid=17733 vsize=340928 CPUtime=2.96 cores=0,2,4,6
/proc/17735/task/17739/stat : 17739 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 11973 0 0 0 295 1 0 0 25 0 5 0 522343672 349110272 40129 33554432000 4194304 4338100 140736114325168 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=17735/tid=17740] ppid=17733 vsize=340928 CPUtime=2.96 cores=0,2,4,6
/proc/17735/task/17740/stat : 17740 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 3389 0 0 0 294 2 0 0 20 0 5 0 522343672 349110272 40129 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826066101 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 11.98
Current children cumulated vsize (KiB) 340928

[startup+6.30058 s]
/proc/loadavg: 7.32 7.89 7.94 9/186 17744
/proc/meminfo: memFree=30063084/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=471496 CPUtime=24.75 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 71225 0 0 0 2452 23 0 0 25 0 5 0 522343660 482811904 70499 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 117874 70499 176 36 0 115779 0
[pid=17735/tid=17737] ppid=17733 vsize=471496 CPUtime=6.12 cores=0,2,4,6
/proc/17735/task/17737/stat : 17737 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 23185 0 0 0 606 6 0 0 25 0 5 0 522343672 482811904 70499 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=17735/tid=17738] ppid=17733 vsize=471496 CPUtime=6.16 cores=0,2,4,6
/proc/17735/task/17738/stat : 17738 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 12565 0 0 0 610 6 0 0 25 0 5 0 522343672 482811904 70499 33554432000 4194304 4338100 140736114325168 18446744073709551615 4285491 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=17735/tid=17739] ppid=17733 vsize=471496 CPUtime=6.16 cores=0,2,4,6
/proc/17735/task/17739/stat : 17739 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 22535 0 0 0 613 3 0 0 25 0 5 0 522343672 482811904 70499 33554432000 4194304 4338100 140736114325168 18446744073709551615 251813635111 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=17735/tid=17740] ppid=17733 vsize=471496 CPUtime=6.16 cores=0,2,4,6
/proc/17735/task/17740/stat : 17740 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 6837 0 0 0 610 6 0 0 25 0 5 0 522343672 482811904 70500 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826066101 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 24.75
Current children cumulated vsize (KiB) 471496

[startup+12.7005 s]
/proc/loadavg: 7.45 7.91 7.95 11/187 17745
/proc/meminfo: memFree=29707804/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=689768 CPUtime=50.34 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 126093 0 0 0 4983 51 0 0 25 0 5 0 522343660 706322432 125367 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 172442 125367 176 36 0 170347 0
[pid=17735/tid=17737] ppid=17733 vsize=689768 CPUtime=12.49 cores=0,2,4,6
/proc/17735/task/17737/stat : 17737 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 42728 0 0 0 1240 9 0 0 25 0 5 0 522343672 706322432 125367 33554432000 4194304 4338100 140736114325168 18446744073709551615 4265498 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=17735/tid=17738] ppid=17733 vsize=689768 CPUtime=12.57 cores=0,2,4,6
/proc/17735/task/17738/stat : 17738 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 22566 0 0 0 1239 18 0 0 25 0 5 0 522343672 706322432 125367 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826054177 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=17735/tid=17739] ppid=17733 vsize=689768 CPUtime=12.57 cores=0,2,4,6
/proc/17735/task/17739/stat : 17739 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 40398 0 0 0 1250 7 0 0 25 0 5 0 522343672 706322432 125367 33554432000 4194304 4338100 140736114325168 18446744073709551615 4256770 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=17735/tid=17740] ppid=17733 vsize=689768 CPUtime=12.56 cores=0,2,4,6
/proc/17735/task/17740/stat : 17740 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 14298 0 0 0 1241 15 0 0 25 0 5 0 522343672 706322432 125367 33554432000 4194304 4338100 140736114325168 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 50.34
Current children cumulated vsize (KiB) 689768

[startup+25.5004 s]
/proc/loadavg: 7.79 7.96 7.96 10/187 17745
/proc/meminfo: memFree=29179320/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=1034640 CPUtime=100.93 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 213738 0 0 0 9962 131 0 0 25 0 5 0 522343660 1059471360 213012 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 258660 213012 176 36 0 256565 0
[pid=17735/tid=17737] ppid=17733 vsize=1034640 CPUtime=25.01 cores=0,2,4,6
/proc/17735/task/17737/stat : 17737 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 78666 0 0 0 2482 19 0 0 25 0 5 0 522343672 1059471360 213012 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=17735/tid=17738] ppid=17733 vsize=1034640 CPUtime=25.35 cores=0,2,4,6
/proc/17735/task/17738/stat : 17738 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 34026 0 0 0 2486 49 0 0 25 0 5 0 522343672 1059471360 213012 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826066101 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=17735/tid=17739] ppid=17733 vsize=1034640 CPUtime=25.11 cores=0,2,4,6
/proc/17735/task/17739/stat : 17739 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 73067 0 0 0 2498 13 0 0 25 0 5 0 522343672 1059471360 213012 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=17735/tid=17740] ppid=17733 vsize=1034640 CPUtime=25.3 cores=0,2,4,6
/proc/17735/task/17740/stat : 17740 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 21876 0 0 0 2483 47 0 0 25 0 5 0 522343672 1059471360 213012 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826066101 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 100.93
Current children cumulated vsize (KiB) 1034640

[startup+51.1011 s]
/proc/loadavg: 7.94 7.98 7.97 9/186 17746
/proc/meminfo: memFree=28235844/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=1646116 CPUtime=203.02 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 365722 0 0 0 20022 280 0 0 25 0 5 0 522343660 1685622784 364996 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 411529 364996 176 36 0 409434 0
[pid=17735/tid=17737] ppid=17733 vsize=1646116 CPUtime=50.41 cores=0,2,4,6
/proc/17735/task/17737/stat : 17737 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 142826 0 0 0 5006 35 0 0 25 0 5 0 522343672 1685622784 364996 33554432000 4194304 4338100 140736114325168 18446744073709551615 4285359 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=17735/tid=17738] ppid=17733 vsize=1646116 CPUtime=50.91 cores=0,2,4,6
/proc/17735/task/17738/stat : 17738 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 53464 0 0 0 4979 112 0 0 25 0 5 0 522343672 1685622784 364997 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826066101 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=17735/tid=17739] ppid=17733 vsize=1646116 CPUtime=50.66 cores=0,2,4,6
/proc/17735/task/17739/stat : 17739 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 129492 0 0 0 5044 22 0 0 25 0 5 0 522343672 1685622784 364997 33554432000 4194304 4338100 140736114325168 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=17735/tid=17740] ppid=17733 vsize=1646116 CPUtime=50.86 cores=0,2,4,6
/proc/17735/task/17740/stat : 17740 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 33838 0 0 0 4979 107 0 0 25 0 5 0 522343672 1685622784 364997 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826066101 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 203.02
Current children cumulated vsize (KiB) 1646116

[startup+102.3 s]
/proc/loadavg: 8.02 8.00 7.98 9/186 17747
/proc/meminfo: memFree=26760584/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=2562724 CPUtime=407.31 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 597019 0 0 0 40103 628 0 0 25 0 5 0 522343660 2624229376 596293 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 640681 596293 176 36 0 638586 0
[pid=17735/tid=17737] ppid=17733 vsize=2562724 CPUtime=101.51 cores=0,2,4,6
/proc/17735/task/17737/stat : 17737 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 246134 0 0 0 10091 60 0 0 25 0 5 0 522343672 2624229376 596293 33554432000 4194304 4338100 140736114325168 18446744073709551615 4240524 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=17735/tid=17738] ppid=17733 vsize=2562724 CPUtime=101.96 cores=0,2,4,6
/proc/17735/task/17738/stat : 17738 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 75533 0 0 0 9933 263 0 0 25 0 5 0 522343672 2624229376 596293 33554432000 4194304 4338100 140736114325168 18446744073709551615 251813936656 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=17735/tid=17739] ppid=17733 vsize=2562724 CPUtime=101.82 cores=0,2,4,6
/proc/17735/task/17739/stat : 17739 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 222591 0 0 0 10141 41 0 0 25 0 5 0 522343672 2624229376 596294 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=17735/tid=17740] ppid=17733 vsize=2562724 CPUtime=101.86 cores=0,2,4,6
/proc/17735/task/17740/stat : 17740 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 46658 0 0 0 9925 261 0 0 25 0 5 0 522343672 2624229376 596294 33554432000 4194304 4338100 140736114325168 18446744073709551615 251813635336 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 407.31
Current children cumulated vsize (KiB) 2562724

[startup+162.3 s]
/proc/loadavg: 8.08 8.02 7.99 9/186 17749
/proc/meminfo: memFree=25417980/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=3393500 CPUtime=646.58 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 802526 0 0 0 63668 990 0 0 25 0 5 0 522343660 3474944000 801800 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 848375 801800 176 36 0 846280 0
[pid=17735/tid=17737] ppid=17733 vsize=3393500 CPUtime=161.39 cores=0,2,4,6
/proc/17735/task/17737/stat : 17737 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 344688 0 0 0 16053 86 0 0 25 0 5 0 522343672 3474944000 801800 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826054299 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=17735/tid=17738] ppid=17733 vsize=3393500 CPUtime=161.84 cores=0,2,4,6
/proc/17735/task/17738/stat : 17738 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 92830 0 0 0 15760 424 0 0 25 0 5 0 522343672 3474944000 801800 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254014 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=17735/tid=17739] ppid=17733 vsize=3393500 CPUtime=161.71 cores=0,2,4,6
/proc/17735/task/17739/stat : 17739 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 301770 0 0 0 16114 57 0 0 25 0 5 0 522343672 3474944000 801800 33554432000 4194304 4338100 140736114325168 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=17735/tid=17740] ppid=17733 vsize=3393500 CPUtime=161.49 cores=0,2,4,6
/proc/17735/task/17740/stat : 17740 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 57135 0 0 0 15728 421 0 0 25 0 5 0 522343672 3474944000 801800 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 646.58
Current children cumulated vsize (KiB) 3393500

[startup+222.3 s]
/proc/loadavg: 8.06 8.03 7.99 9/186 17751
/proc/meminfo: memFree=24262756/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=4161352 CPUtime=885.58 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 997161 0 0 0 87270 1288 0 0 25 0 5 0 522343660 4261224448 996435 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 1040338 996435 176 36 0 1038243 0
[pid=17735/tid=17737] ppid=17733 vsize=4161352 CPUtime=221.23 cores=0,2,4,6
/proc/17735/task/17737/stat : 17737 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 439737 0 0 0 22013 110 0 0 25 0 5 0 522343672 4261224448 996435 33554432000 4194304 4338100 140736114325168 18446744073709551615 251813635463 0 0 4096 0 0 0 0 -1 6 0 0 0

################
# More data... #
################

/proc/loadavg: 8.14 8.11 8.03 9/186 17765
/proc/meminfo: memFree=16261120/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=8922128 CPUtime=3035.68 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 2186973 0 0 0 300105 3463 0 0 25 0 5 0 522343660 9136259072 2186247 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 2230532 2186247 176 36 0 2228437 0
[pid=17735/tid=17737] ppid=17733 vsize=8922128 CPUtime=758.04 cores=0,2,4,6
/proc/17735/task/17737/stat : 17737 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 1022675 0 0 0 75525 279 0 0 25 0 5 0 522343672 9136259072 2186247 33554432000 4194304 4338100 140736114325168 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=17735/tid=17738] ppid=17733 vsize=8922128 CPUtime=757.34 cores=0,2,4,6
/proc/17735/task/17738/stat : 17738 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 177597 0 0 0 74333 1401 0 0 25 0 5 0 522343672 9136259072 2186247 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=17735/tid=17739] ppid=17733 vsize=8922128 CPUtime=760.5 cores=0,2,4,6
/proc/17735/task/17739/stat : 17739 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 900064 0 0 0 75878 172 0 0 25 0 5 0 522343672 9136259072 2186247 33554432000 4194304 4338100 140736114325168 18446744073709551615 4206902 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=17735/tid=17740] ppid=17733 vsize=8922128 CPUtime=759.65 cores=0,2,4,6
/proc/17735/task/17740/stat : 17740 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 80534 0 0 0 74357 1608 0 0 25 0 5 0 522343672 9136259072 2186247 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3035.68
Current children cumulated vsize (KiB) 8922128

[startup+822.301 s]
/proc/loadavg: 8.18 8.13 8.04 9/186 17767
/proc/meminfo: memFree=15610920/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=9483136 CPUtime=3274.58 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 2328791 0 0 0 323660 3798 0 0 25 0 5 0 522343660 9710731264 2328065 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 2370784 2328065 176 36 0 2368689 0
[pid=17735/tid=17737] ppid=17733 vsize=9483136 CPUtime=817.82 cores=0,2,4,6
/proc/17735/task/17737/stat : 17737 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 1087302 0 0 0 81485 297 0 0 25 0 5 0 522343672 9710731264 2328065 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=17735/tid=17738] ppid=17733 vsize=9483136 CPUtime=817.13 cores=0,2,4,6
/proc/17735/task/17738/stat : 17738 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 178032 0 0 0 80163 1550 0 0 25 0 5 0 522343672 9710731264 2328065 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=17735/tid=17739] ppid=17733 vsize=9483136 CPUtime=820.4 cores=0,2,4,6
/proc/17735/task/17739/stat : 17739 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 976338 0 0 0 81852 188 0 0 25 0 5 0 522343672 9710731264 2328065 33554432000 4194304 4338100 140736114325168 18446744073709551615 251813635199 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=17735/tid=17740] ppid=17733 vsize=9483136 CPUtime=819.08 cores=0,2,4,6
/proc/17735/task/17740/stat : 17740 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 81016 0 0 0 80148 1760 0 0 25 0 5 0 522343672 9710731264 2328065 33554432000 4194304 4338100 140736114325168 18446744073709551615 4241248 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3274.58
Current children cumulated vsize (KiB) 9483136

[startup+882.301 s]
/proc/loadavg: 8.10 8.12 8.04 9/186 17769
/proc/meminfo: memFree=14916180/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=9990836 CPUtime=3513.62 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 2457678 0 0 0 347248 4114 0 0 25 0 5 0 522343660 10230616064 2456952 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 2497709 2456952 176 36 0 2495614 0
[pid=17735/tid=17737] ppid=17733 vsize=9990836 CPUtime=877.61 cores=0,2,4,6
/proc/17735/task/17737/stat : 17737 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 1159737 0 0 0 87443 318 0 0 25 0 5 0 522343672 10230616064 2456952 33554432000 4194304 4338100 140736114325168 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=17735/tid=17738] ppid=17733 vsize=9990836 CPUtime=876.79 cores=0,2,4,6
/proc/17735/task/17738/stat : 17738 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 178498 0 0 0 85985 1694 0 0 25 0 5 0 522343672 10230616064 2456952 33554432000 4194304 4338100 140736114325168 18446744073709551615 4241488 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=17735/tid=17739] ppid=17733 vsize=9990836 CPUtime=880.28 cores=0,2,4,6
/proc/17735/task/17739/stat : 17739 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 1031886 0 0 0 87830 198 0 0 25 0 5 0 522343672 10230616064 2456952 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=17735/tid=17740] ppid=17733 vsize=9990836 CPUtime=878.79 cores=0,2,4,6
/proc/17735/task/17740/stat : 17740 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 81454 0 0 0 85977 1902 0 0 25 0 5 0 522343672 10230616064 2456952 33554432000 4194304 4338100 140736114325168 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3513.62
Current children cumulated vsize (KiB) 9990836

[startup+942.307 s]
/proc/loadavg: 8.08 8.11 8.03 9/186 17770
/proc/meminfo: memFree=14072024/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=10401700 CPUtime=3752.02 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 2562625 0 0 0 370940 4262 0 0 25 0 5 0 522343660 10651340800 2561899 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 2600425 2561899 176 36 0 2598330 0
[pid=17735/tid=17737] ppid=17733 vsize=10401700 CPUtime=937.29 cores=0,2,4,6
/proc/17735/task/17737/stat : 17737 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 1226264 0 0 0 93390 339 0 0 25 0 5 0 522343672 10651340800 2561899 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=17735/tid=17738] ppid=17733 vsize=10401700 CPUtime=936.09 cores=0,2,4,6
/proc/17735/task/17738/stat : 17738 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 178964 0 0 0 91874 1735 0 0 18 0 5 0 522343672 10651340800 2561899 33554432000 4194304 4338100 140736114325168 18446744073709551615 4284277 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=17735/tid=17739] ppid=17733 vsize=10401700 CPUtime=939.89 cores=0,2,4,6
/proc/17735/task/17739/stat : 17739 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 1068979 0 0 0 93784 205 0 0 25 0 5 0 522343672 10651340800 2561899 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=17735/tid=17740] ppid=17733 vsize=10401700 CPUtime=938.59 cores=0,2,4,6
/proc/17735/task/17740/stat : 17740 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 82315 0 0 0 91879 1980 0 0 25 0 5 0 522343672 10651340800 2561899 33554432000 4194304 4338100 140736114325168 18446744073709551615 4286480 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3752.02
Current children cumulated vsize (KiB) 10401700

[startup+1002.3 s]
/proc/loadavg: 8.06 8.10 8.03 9/186 17771
/proc/meminfo: memFree=13664944/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=10674780 CPUtime=3990.33 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 2628292 0 0 0 394681 4352 0 0 25 0 5 0 522343660 10930974720 2627566 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 2668695 2627566 176 36 0 2666600 0
[pid=17735/tid=17737] ppid=17733 vsize=10674780 CPUtime=996.8 cores=0,2,4,6
/proc/17735/task/17737/stat : 17737 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 1261995 0 0 0 99329 351 0 0 25 0 5 0 522343672 10930974720 2627566 33554432000 4194304 4338100 140736114325168 18446744073709551615 4253954 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=17735/tid=17738] ppid=17733 vsize=10674780 CPUtime=995.29 cores=0,2,4,6
/proc/17735/task/17738/stat : 17738 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 183899 0 0 0 97778 1751 0 0 18 0 5 0 522343672 10930974720 2627566 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826066101 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=17735/tid=17739] ppid=17733 vsize=10674780 CPUtime=999.72 cores=0,2,4,6
/proc/17735/task/17739/stat : 17739 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 1086320 0 0 0 99762 210 0 0 25 0 5 0 522343672 10930974720 2627566 33554432000 4194304 4338100 140736114325168 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=17735/tid=17740] ppid=17733 vsize=10674780 CPUtime=998.38 cores=0,2,4,6
/proc/17735/task/17740/stat : 17740 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 89975 0 0 0 97800 2038 0 0 25 0 5 0 522343672 10930974720 2627566 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826066101 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3990.33
Current children cumulated vsize (KiB) 10674780

[startup+1062.3 s]
/proc/loadavg: 8.05 8.09 8.03 9/186 17773
/proc/meminfo: memFree=13114308/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=11019068 CPUtime=4228.87 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 2690123 0 0 0 418221 4666 0 0 25 0 5 0 522343660 11283525632 2689397 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 2754767 2689397 176 36 0 2752672 0
[pid=17735/tid=17737] ppid=17733 vsize=11019068 CPUtime=1056.39 cores=0,2,4,6
/proc/17735/task/17737/stat : 17737 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 1294632 0 0 0 105273 366 0 0 25 0 5 0 522343672 11283525632 2689397 33554432000 4194304 4338100 140736114325168 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=17735/tid=17738] ppid=17733 vsize=11019068 CPUtime=1054.97 cores=0,2,4,6
/proc/17735/task/17738/stat : 17738 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 191358 0 0 0 103607 1890 0 0 25 0 5 0 522343672 11283525632 2689398 33554432000 4194304 4338100 140736114325168 18446744073709551615 251813635315 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=17735/tid=17739] ppid=17733 vsize=11019068 CPUtime=1059.49 cores=0,2,4,6
/proc/17735/task/17739/stat : 17739 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 1098722 0 0 0 105736 213 0 0 25 0 5 0 522343672 11283525632 2689398 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=17735/tid=17740] ppid=17733 vsize=11019068 CPUtime=1057.86 cores=0,2,4,6
/proc/17735/task/17740/stat : 17740 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 99308 0 0 0 103592 2194 0 0 25 0 5 0 522343672 11283525632 2689398 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4228.87
Current children cumulated vsize (KiB) 11019068

[startup+1122.3 s]
/proc/loadavg: 8.05 8.09 8.03 9/186 17775
/proc/meminfo: memFree=12498924/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=11305340 CPUtime=4467.72 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 2779809 0 0 0 441758 5014 0 0 25 0 5 0 522343660 11576668160 2779083 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 2826335 2779083 176 36 0 2824240 0
[pid=17735/tid=17737] ppid=17733 vsize=11305340 CPUtime=1116.08 cores=0,2,4,6
/proc/17735/task/17737/stat : 17737 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 1334748 0 0 0 111227 381 0 0 25 0 5 0 522343672 11576668160 2779083 33554432000 4194304 4338100 140736114325168 18446744073709551615 251813635088 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=17735/tid=17738] ppid=17733 vsize=11305340 CPUtime=1114.74 cores=0,2,4,6
/proc/17735/task/17738/stat : 17738 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 199157 0 0 0 109426 2048 0 0 25 0 5 0 522343672 11576668160 2779083 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826066101 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=17735/tid=17739] ppid=17733 vsize=11305340 CPUtime=1119.26 cores=0,2,4,6
/proc/17735/task/17739/stat : 17739 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 1130955 0 0 0 111706 220 0 0 25 0 5 0 522343672 11576668160 2779083 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254321 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=17735/tid=17740] ppid=17733 vsize=11305340 CPUtime=1117.48 cores=0,2,4,6
/proc/17735/task/17740/stat : 17740 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 108846 0 0 0 109387 2361 0 0 25 0 5 0 522343672 11576668160 2779083 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826066101 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4467.72
Current children cumulated vsize (KiB) 11305340

[startup+1182.3 s]
/proc/loadavg: 8.06 8.08 8.03 9/186 17777
/proc/meminfo: memFree=11977424/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=11773672 CPUtime=4706.39 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) S 17733 17735 17262 0 -1 4202496 2850497 0 0 0 465416 5223 0 0 25 0 5 0 522343660 12056240128 2849771 33554432000 4194304 4338100 140736114325168 18446744073709551615 251826042853 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 2943418 2849771 176 36 0 2941323 0
[pid=17735/tid=17737] ppid=17733 vsize=11773672 CPUtime=1175.68 cores=0,2,4,6
/proc/17735/task/17737/stat : 17737 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 1380901 0 0 0 117171 397 0 0 25 0 5 0 522343672 12056240128 2849771 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254321 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=17735/tid=17738] ppid=17733 vsize=11773672 CPUtime=1174.45 cores=0,2,4,6
/proc/17735/task/17738/stat : 17738 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 204504 0 0 0 115290 2155 0 0 25 0 5 0 522343672 12056240128 2849771 33554432000 4194304 4338100 140736114325168 18446744073709551615 251813635509 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=17735/tid=17739] ppid=17733 vsize=11773672 CPUtime=1179.14 cores=0,2,4,6
/proc/17735/task/17739/stat : 17739 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 1144813 0 0 0 117691 223 0 0 25 0 5 0 522343672 12056240128 2849771 33554432000 4194304 4338100 140736114325168 18446744073709551615 251813936630 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=17735/tid=17740] ppid=17733 vsize=11773672 CPUtime=1176.96 cores=0,2,4,6
/proc/17735/task/17740/stat : 17740 (ttsth-5-4) R 17733 17735 17262 0 -1 4202560 114176 0 0 0 115251 2445 0 0 22 0 5 0 522343672 12056240128 2849771 33554432000 4194304 4338100 140736114325168 18446744073709551615 4254816 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4706.39
Current children cumulated vsize (KiB) 11773672



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1205.8 s]
/proc/loadavg: 7.72 8.01 8.01 2/172 17795
/proc/meminfo: memFree=19504384/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=0 CPUtime=4800.01 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) R 17733 17735 17262 0 -1 4203524 2878801 0 0 0 474682 5319 0 0 18 0 1 0 522343660 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 4800.01
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1206.1 s]
/proc/loadavg: 7.72 8.01 8.01 2/172 17795
/proc/meminfo: memFree=19504384/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=0 CPUtime=4800.31 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) R 17733 17735 17262 0 -1 4203524 2878801 0 0 0 474682 5349 0 0 18 0 1 0 522343660 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 4800.31
Current children cumulated vsize (KiB) 0

[startup+1206.2 s]
/proc/loadavg: 7.72 8.01 8.01 2/173 17796
/proc/meminfo: memFree=30081724/32950928 swapFree=67111448/67111528
[pid=17735] ppid=17733 vsize=0 CPUtime=4800.41 cores=0,2,4,6
/proc/17735/stat : 17735 (ttsth-5-4) R 17733 17735 17262 0 -1 4203524 2878801 0 0 0 474682 5359 0 0 18 0 1 0 522343660 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/17735/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 4800.41
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1206.23
CPU time (s): 4800.44
CPU user time (s): 4746.82
CPU system time (s): 53.6198
CPU usage (%): 397.971
Max. virtual memory (cumulated for all children) (KiB): 11881476

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4746.82
system time used= 53.6198
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2878801
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 24717
involuntary context switches= 20974

runsolver used 1.9427 second user time and 4.5963 second system time

The end

Launcher Data 

Begin job on node126 at 2011-04-29 22:04:10
IDJOB=3266930
IDBENCH=88220
IDSOLVER=1654
FILE ID=node126/3266930-1304107450
RUNJOBID= node126-1304106237-17608
PBS_JOBID= 13170635
Free space on /tmp= 71588 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m27-q255-n25-p15-s2076598.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3266930-1304107450/watcher-3266930-1304107450 -o /tmp/evaluation-result-3266930-1304107450/solver-3266930-1304107450 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3266930-1304107450.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f5a22d1154b549fd98215f1710a15f46
RANDOM SEED=454794093

node126.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30577352 kB
Buffers:        253864 kB
Cached:        1415784 kB
SwapCached:         76 kB
Active:         464796 kB
Inactive:      1595408 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30577352 kB
SwapTotal:    67111528 kB
SwapFree:     67111448 kB
Dirty:           12996 kB
Writeback:           0 kB
AnonPages:      390540 kB
Mapped:          15196 kB
Slab:           247604 kB
PageTables:       5116 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   529840 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71600 MiB
End job on node126 at 2011-04-29 22:24:17