Trace number 3264950

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.33 1225.92

General information on the benchmark

NameSAT05/crafted/jarvisalo05/mod2c-rand3bip-sat/
mod2c-rand3bip-sat-240-2.shuffled-as.sat05-2519.cnf
MD5SUMb6a88c8ae73b20cfc873207d1a0caa09
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark139.097
Satisfiable
(Un)Satisfiability was proved
Number of variables338
Number of clauses2376
Sum of the clauses size10480
Maximum clause length6
Minimum clause length4
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 41520
Number of clauses of size 5736
Number of clauses of size over 5120

Solver Data 

0.00/0.00	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3264950-1304100670.cnf 
0.00/0.00	c Using SATTIMEOUT of 4800
0.00/0.00	c Using SATRAM of 15500
0.00/0.00	c Size of subset node = 32
0.00/0.00	c Variables = 338
0.00/0.00	c Clauses = 2376
0.00/0.00	c Literals = 10480
0.00/0.00	c Reduced size:
0.00/0.00	c Variables = 338
0.00/0.00	c Clauses = 2376
0.00/0.00	c Literals = 10480
0.00/0.00	c a2
0.00/0.00	c a1
0.00/0.07	c Independent sub-problems = 1
0.00/0.07	c Sub-problem with 338 variables and 2376 clauses
0.00/0.07	c using permutation method 4
0.00/0.07	c count = 2
0.00/0.07	c Ordering finished 0.070 seconds
0.00/0.07	c mapped 0.070 seconds
0.00/0.07	c permuted 0.070 seconds
0.00/0.07	c about to sort
0.00/0.07	c sorted
0.00/0.08	c clauses now 2376
0.00/0.08	c init finished 0.070 seconds
0.00/0.08	c tree built 0.070 seconds
0.00/0.08	c 
0.00/0.08	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.00/0.08	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.00/0.08	c ................................ | ..........                    |                                  0.1         0.0             
0.00/0.08	c ................................ | ..........                  1 |                                  0.1        10.0             
0.00/0.08	c ................................ | ..........                  2 |                                  0.1        20.0             
0.00/0.08	c ................................ | ..........                  3 |                                  0.1        30.0             
0.00/0.08	c ................................ | ..........                  4 |                                  0.1        40.0             
0.00/0.08	c ................................ | ..........                  5 |                                  0.1        50.0             
0.00/0.08	c ................................ | ..........               1000 | 114          661                 0.1     10000.0        11955
95.65/24.20	c ................................ | ..........            1387000 | 376262       576261             96.0     14440.4     20838292
191.83/48.45	c ................................ | ..........            2679000 | 729077       1102457           192.0     13950.2     37456347
287.67/72.69	c ................................ | ..........            3709000 | 1017692      1528490           288.1     12875.3     44734162
384.02/96.93	c ................................ | ..........            4495000 | 1243945      1842735           384.1     11704.2     50418324
479.67/121.29	c ................................ | *.........            5386000 | 1497074      2207688           480.0     11220.6     55649872
575.70/145.67	c ................................ | *.........            6249000 | 1741469      2549923           576.0     10849.0     60261779
671.75/169.98	c ................................ | *.........            6987000 | 1958690      2846618           672.1     10396.1     63915976
767.77/194.36	c ................................ | *.........            7738000 | 2176552      3148124           768.0     10075.3     66771386
863.81/218.67	c ................................ | *.........            8494000 | 2394682      3449738           864.1      9830.0     70364096
959.94/243.02	c ................................ | **........            9181000 | 2593921      3729884           960.0      9563.2     74676568
1055.81/267.37	c ................................ | **........            9812000 | 2777989      3982683          1056.1      9290.8     77779642
1152.03/291.72	c ................................ | **........           10519000 | 2981537      4266966          1152.1      9130.4     82177964
1248.03/316.10	c ................................ | **........           11180000 | 3172731      4524720          1248.0      8958.2     83800856
1343.83/340.36	c ................................ | **........           11817000 | 3356323      4778331          1344.1      8791.8     87596428
1439.93/364.62	c ................................ | ***.......           12457000 | 3541890      5036584          1440.0      8650.5     90910967
1536.05/389.20	c ................................ | ***.......           13128000 | 3735564      5292936          1536.0      8546.6     91709339
1632.09/413.60	c ................................ | ***.......           13762000 | 3921129      5539898          1632.1      8432.1     93439610
1727.92/438.23	c ................................ | ***.......           14403000 | 4107373      5798942          1728.0      8335.0     97163137
1823.67/462.79	c ................................ | ***.......           15062000 | 4298289      6057463          1824.0      8257.5     99922460
1919.98/487.13	c ................................ | ****......           15670000 | 4477787      6299334          1920.1      8160.9    104091245
2016.02/511.82	c ................................ | ****......           16239000 | 4645720      6529251          2016.1      8054.7    107317858
2111.90/536.25	c ................................ | ****......           16867000 | 4830058      6773228          2112.1      7985.9    111046299
2207.71/560.68	c ................................ | ****......           17449000 | 5002046      6993569          2208.1      7902.4    112077014
2303.73/585.16	c ................................ | ****......           17984000 | 5159870      7210443          2304.0      7805.6    113458929
2400.00/609.73	c ................................ | *****.....           18570000 | 5332045      7440960          2400.1      7737.1    116066025
2495.92/634.14	c ................................ | *****.....           19140000 | 5498442      7663728          2496.1      7668.0    118883443
2591.70/658.57	c ................................ | *****.....           19676000 | 5652719      7879563          2592.0      7591.0    119923644
2688.04/683.00	c ................................ | *****.....           20256000 | 5814909      8100681          2688.1      7535.5    121415037
2783.89/707.54	c ................................ | *****.....           20822000 | 5976505      8319424          2784.0      7479.1    123613296
2880.06/732.00	c ................................ | ******....           21381000 | 6134302      8537557          2880.1      7423.7    126952626
2975.95/756.63	c ................................ | ******....           21951000 | 6294499      8757721          2976.1      7375.8    129349720
3071.82/781.06	c ................................ | ******....           22494000 | 6452326      8970705          3072.1      7322.1    130706061
3167.72/805.69	c ................................ | ******....           23071000 | 6617230      9191938          3168.1      7282.3    132445678
3263.95/830.15	c ................................ | ******....           23606000 | 6772299      9406443          3264.2      7231.9    134879997
3361.27/854.78	c ................................ | *******...           24170000 | 6935663      9630861          3361.5      7190.2    138864862
3455.79/878.77	c ................................ | *******...           24739000 | 7100963      9862214          3456.1      7158.1    140399050
3551.95/903.51	c ................................ | *******...           25309000 | 7267575      10093843         3552.0      7125.3    142143518
3647.73/927.97	c ................................ | *******...           25874000 | 7431532      10324308         3648.0      7092.6    145055116
3743.76/952.56	c ................................ | *******...           26409000 | 7587707      10541820         3744.0      7053.7    145736243
3839.95/977.42	c ................................ | ********..           26985000 | 7755615      10767171         3840.1      7027.2    147384778
3935.83/1002.27	c ................................ | ********..           27514000 | 7910877      10985717         3936.1      6990.2    148975271
4031.80/1026.59	c ................................ | ********..           28094000 | 8079688      11211089         4032.2      6967.5    151157207
4127.89/1051.04	c ................................ | ********..           28598000 | 8228009      11419261         4128.1      6927.7    153323038
4223.89/1075.56	c ................................ | ********..           29141000 | 8386542      11639327         4224.1      6898.8    153472795
4319.77/1100.89	c ................................ | *********.           29730000 | 8556567      11871623         4320.1      6881.7    153786857
4415.88/1125.85	c ................................ | *********.           30342000 | 8731567      12128641         4416.1      6870.8    157025347
4511.99/1150.93	c ................................ | *********.           31024000 | 8926093      12403631         4512.1      6875.7    159389247
4607.79/1175.68	c ................................ | *********.           31672000 | 9111145      12668158         4608.1      6873.1    161844598
4704.01/1200.61	c ................................ | *********.           32285000 | 9286474      12922175         4704.1      6863.2    162042178
4800.10/1225.71	c CPU limit (4800) reached
4800.10/1225.71	c Time = 4800.11 seconds
4800.10/1225.71	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3264950-1304100670/watcher-3264950-1304100670 -o /tmp/evaluation-result-3264950-1304100670/solver-3264950-1304100670 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3264950-1304100670.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.99 7.20 7.73 2/176 7090
/proc/meminfo: memFree=19936340/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=9640 CPUtime=0 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) R 7085 7090 6445 0 -1 4202496 486 0 0 0 0 0 0 0 23 0 1 0 521666798 9871360 428 33554432000 4194304 4338100 140735122756960 18446744073709551615 237278274631 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/7090/statm: 2410 428 152 36 0 315 0

[startup+0.0920581 s]
/proc/loadavg: 4.99 7.20 7.73 2/176 7090
/proc/meminfo: memFree=19936340/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=198376 CPUtime=0.2 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 2422 0 0 0 20 0 0 0 23 0 5 0 521666798 203137024 2327 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 66825 3958 175 36 0 64730 0
Current children cumulated CPU time (s) 0.2
Current children cumulated vsize (KiB) 198376

[startup+0.903475 s]
/proc/loadavg: 4.99 7.20 7.73 2/176 7090
/proc/meminfo: memFree=19936340/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=311616 CPUtime=3.69 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 16116 0 0 0 364 5 0 0 23 0 5 0 521666798 319094784 16021 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 77904 16021 175 36 0 75809 0
Current children cumulated CPU time (s) 3.69
Current children cumulated vsize (KiB) 311616

[startup+1.09056 s]
/proc/loadavg: 4.99 7.20 7.73 2/176 7090
/proc/meminfo: memFree=19936340/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=321000 CPUtime=4.08 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 17359 0 0 0 403 5 0 0 23 0 5 0 521666798 328704000 17264 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 80250 17264 175 36 0 78155 0
Current children cumulated CPU time (s) 4.08
Current children cumulated vsize (KiB) 321000

[startup+1.10413 s]
/proc/loadavg: 4.99 7.20 7.73 2/176 7090
/proc/meminfo: memFree=19936340/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=324128 CPUtime=4.49 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 18667 0 0 0 443 6 0 0 23 0 5 0 521666798 331907072 18572 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 81032 18572 175 36 0 78937 0
Current children cumulated CPU time (s) 4.49
Current children cumulated vsize (KiB) 324128

[startup+1.6916 s]
/proc/loadavg: 4.99 7.20 7.73 2/176 7090
/proc/meminfo: memFree=19936340/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=349412 CPUtime=6.56 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 24217 0 0 0 647 9 0 0 23 0 5 0 521666798 357797888 24122 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 87353 24122 175 36 0 85258 0
Current children cumulated CPU time (s) 6.56
Current children cumulated vsize (KiB) 349412

[startup+3.10027 s]
/proc/loadavg: 5.31 7.23 7.73 10/186 7100
/proc/meminfo: memFree=19672668/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=393592 CPUtime=12.1 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 36789 0 0 0 1193 17 0 0 23 0 5 0 521666798 403038208 36694 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 98398 36694 175 36 0 96303 0
[pid=7090/tid=7097] ppid=7085 vsize=393592 CPUtime=3 cores=0,2,4,6
/proc/7090/task/7097/stat : 7097 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 5857 0 0 0 294 6 0 0 19 0 5 0 521666806 403038208 36695 33554432000 4194304 4338100 140735122756960 18446744073709551615 4242504 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=7090/tid=7098] ppid=7085 vsize=393592 CPUtime=3 cores=0,2,4,6
/proc/7090/task/7098/stat : 7098 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 5587 0 0 0 297 3 0 0 25 0 5 0 521666806 403038208 36695 33554432000 4194304 4338100 140735122756960 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=7090/tid=7099] ppid=7085 vsize=393592 CPUtime=3.01 cores=0,2,4,6
/proc/7090/task/7099/stat : 7099 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 17369 0 0 0 299 2 0 0 25 0 5 0 521666806 403038208 36695 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254326 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=7090/tid=7100] ppid=7085 vsize=393592 CPUtime=2.99 cores=0,2,4,6
/proc/7090/task/7100/stat : 7100 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 6507 0 0 0 294 5 0 0 20 0 5 0 521666806 403038208 36695 33554432000 4194304 4338100 140735122756960 18446744073709551615 4242665 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 12.1
Current children cumulated vsize (KiB) 393592

[startup+6.3007 s]
/proc/loadavg: 5.31 7.23 7.73 10/186 7100
/proc/meminfo: memFree=19383368/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=514388 CPUtime=24.82 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 67569 0 0 0 2448 34 0 0 23 0 5 0 521666798 526733312 67446 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 128597 67446 176 36 0 126502 0
[pid=7090/tid=7097] ppid=7085 vsize=514388 CPUtime=6.17 cores=0,2,4,6
/proc/7090/task/7097/stat : 7097 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 11089 0 0 0 606 11 0 0 18 0 5 0 521666806 526733312 67446 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254352 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=7090/tid=7098] ppid=7085 vsize=514388 CPUtime=6.19 cores=0,2,4,6
/proc/7090/task/7098/stat : 7098 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 10530 0 0 0 612 7 0 0 22 0 5 0 521666806 526733312 67446 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=7090/tid=7099] ppid=7085 vsize=514388 CPUtime=6.2 cores=0,2,4,6
/proc/7090/task/7099/stat : 7099 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 32242 0 0 0 616 4 0 0 25 0 5 0 521666806 526733312 67447 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=7090/tid=7100] ppid=7085 vsize=514388 CPUtime=6.14 cores=0,2,4,6
/proc/7090/task/7100/stat : 7100 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 12238 0 0 0 604 10 0 0 19 0 5 0 521666806 526733312 67448 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254294 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.82
Current children cumulated vsize (KiB) 514388

[startup+12.7006 s]
/proc/loadavg: 5.80 7.27 7.74 11/186 7101
/proc/meminfo: memFree=18969016/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=685708 CPUtime=50.29 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 109580 0 0 0 4960 69 0 0 23 0 5 0 521666798 702164992 109449 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 171427 109449 176 36 0 169332 0
[pid=7090/tid=7097] ppid=7085 vsize=685708 CPUtime=12.53 cores=0,2,4,6
/proc/7090/task/7097/stat : 7097 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 18306 0 0 0 1231 22 0 0 18 0 5 0 521666806 702164992 109450 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=7090/tid=7098] ppid=7085 vsize=685708 CPUtime=12.58 cores=0,2,4,6
/proc/7090/task/7098/stat : 7098 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 17292 0 0 0 1241 17 0 0 25 0 5 0 521666806 702164992 109450 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=7090/tid=7099] ppid=7085 vsize=685708 CPUtime=12.6 cores=0,2,4,6
/proc/7090/task/7099/stat : 7099 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 53329 0 0 0 1253 7 0 0 25 0 5 0 521666806 702164992 109450 33554432000 4194304 4338100 140735122756960 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=7090/tid=7100] ppid=7085 vsize=685708 CPUtime=12.47 cores=0,2,4,6
/proc/7090/task/7100/stat : 7100 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 19184 0 0 0 1225 22 0 0 19 0 5 0 521666806 702164992 109450 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254291 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.29
Current children cumulated vsize (KiB) 685708

[startup+25.5063 s]
/proc/loadavg: 6.22 7.31 7.75 10/186 7101
/proc/meminfo: memFree=18305616/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=943236 CPUtime=101.21 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 176276 0 0 0 9995 126 0 0 23 0 5 0 521666798 965873664 176145 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 235809 176145 176 36 0 233714 0
[pid=7090/tid=7097] ppid=7085 vsize=943236 CPUtime=25.24 cores=0,2,4,6
/proc/7090/task/7097/stat : 7097 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 30354 0 0 0 2484 40 0 0 18 0 5 0 521666806 965873664 176145 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=7090/tid=7098] ppid=7085 vsize=943236 CPUtime=25.35 cores=0,2,4,6
/proc/7090/task/7098/stat : 7098 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 26572 0 0 0 2503 32 0 0 25 0 5 0 521666806 965873664 176145 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=7090/tid=7099] ppid=7085 vsize=943236 CPUtime=25.39 cores=0,2,4,6
/proc/7090/task/7099/stat : 7099 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 87194 0 0 0 2526 13 0 0 25 0 5 0 521666806 965873664 176145 33554432000 4194304 4338100 140735122756960 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=7090/tid=7100] ppid=7085 vsize=943236 CPUtime=25.13 cores=0,2,4,6
/proc/7090/task/7100/stat : 7100 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 30686 0 0 0 2473 40 0 0 18 0 5 0 521666806 965873664 176145 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.21
Current children cumulated vsize (KiB) 943236

[startup+51.1007 s]
/proc/loadavg: 6.83 7.36 7.75 10/186 7102
/proc/meminfo: memFree=17260284/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=1504996 CPUtime=202.54 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 319563 0 0 0 20007 247 0 0 23 0 5 0 521666798 1541115904 319433 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 376249 319433 176 36 0 374154 0
[pid=7090/tid=7097] ppid=7085 vsize=1504996 CPUtime=50.48 cores=0,2,4,6
/proc/7090/task/7097/stat : 7097 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 55003 0 0 0 4965 83 0 0 18 0 5 0 521666806 1541115904 319433 33554432000 4194304 4338100 140735122756960 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=7090/tid=7098] ppid=7085 vsize=1504996 CPUtime=50.73 cores=0,2,4,6
/proc/7090/task/7098/stat : 7098 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 46609 0 0 0 5012 61 0 0 20 0 5 0 521666806 1541115904 319433 33554432000 4194304 4338100 140735122756960 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=7090/tid=7099] ppid=7085 vsize=1504996 CPUtime=50.96 cores=0,2,4,6
/proc/7090/task/7099/stat : 7099 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 159434 0 0 0 5071 25 0 0 25 0 5 0 521666806 1541115904 319433 33554432000 4194304 4338100 140735122756960 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=7090/tid=7100] ppid=7085 vsize=1504996 CPUtime=50.26 cores=0,2,4,6
/proc/7090/task/7100/stat : 7100 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 57048 0 0 0 4950 76 0 0 19 0 5 0 521666806 1541115904 319433 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 202.54
Current children cumulated vsize (KiB) 1504996

[startup+102.301 s]
/proc/loadavg: 7.39 7.44 7.76 10/186 7103
/proc/meminfo: memFree=15946368/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=1945548 CPUtime=405.25 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 430530 0 0 0 40099 426 0 0 23 0 5 0 521666798 1992241152 430399 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 486387 430399 176 36 0 484292 0
[pid=7090/tid=7097] ppid=7085 vsize=1945548 CPUtime=101.04 cores=0,2,4,6
/proc/7090/task/7097/stat : 7097 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 71718 0 0 0 9963 141 0 0 18 0 5 0 521666806 1992241152 430400 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=7090/tid=7098] ppid=7085 vsize=1945548 CPUtime=101.56 cores=0,2,4,6
/proc/7090/task/7098/stat : 7098 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 54390 0 0 0 10046 110 0 0 18 0 5 0 521666806 1992241152 430400 33554432000 4194304 4338100 140735122756960 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=7090/tid=7099] ppid=7085 vsize=1945548 CPUtime=101.83 cores=0,2,4,6
/proc/7090/task/7099/stat : 7099 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 232203 0 0 0 10144 39 0 0 25 0 5 0 521666806 1992241152 430400 33554432000 4194304 4338100 140735122756960 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=7090/tid=7100] ppid=7085 vsize=1945548 CPUtime=100.7 cores=0,2,4,6
/proc/7090/task/7100/stat : 7100 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 70750 0 0 0 9936 134 0 0 22 0 5 0 521666806 1992241152 430400 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 405.25
Current children cumulated vsize (KiB) 1945548

[startup+162.301 s]
/proc/loadavg: 7.71 7.51 7.76 10/186 7105
/proc/meminfo: memFree=14935120/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=2337372 CPUtime=641.61 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 527887 0 0 0 63534 627 0 0 23 0 5 0 521666798 2393468928 527756 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 584343 527756 176 36 0 582248 0
[pid=7090/tid=7097] ppid=7085 vsize=2337372 CPUtime=159.86 cores=0,2,4,6
/proc/7090/task/7097/stat : 7097 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 97540 0 0 0 15770 216 0 0 18 0 5 0 521666806 2393468928 527756 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=7090/tid=7098] ppid=7085 vsize=2337372 CPUtime=160.54 cores=0,2,4,6
/proc/7090/task/7098/stat : 7098 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 57937 0 0 0 15882 172 0 0 18 0 5 0 521666806 2393468928 527756 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=7090/tid=7099] ppid=7085 vsize=2337372 CPUtime=161.18 cores=0,2,4,6
/proc/7090/task/7099/stat : 7099 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 296183 0 0 0 16067 51 0 0 25 0 5 0 521666806 2393468928 527756 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=7090/tid=7100] ppid=7085 vsize=2337372 CPUtime=159.89 cores=0,2,4,6
/proc/7090/task/7100/stat : 7100 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 74757 0 0 0 15804 185 0 0 19 0 5 0 521666806 2393468928 527756 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 641.61
Current children cumulated vsize (KiB) 2337372

[startup+222.3 s]
/proc/loadavg: 7.76 7.56 7.76 10/186 7106
/proc/meminfo: memFree=14205604/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=2615396 CPUtime=878.5 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 604500 0 0 0 86956 894 0 0 23 0 5 0 521666798 2678165504 604369 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 653849 604369 176 36 0 651754 0
[pid=7090/tid=7097] ppid=7085 vsize=2615396 CPUtime=218.93 cores=0,2,4,6
/proc/7090/task/7097/stat : 7097 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 102829 0 0 0 21587 306 0 0 25 0 5 0 521666806 2678165504 604369 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 2 0 0 0

################
# More data... #
################

/proc/meminfo: memFree=9527036/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=4875732 CPUtime=2998.27 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 1142120 0 0 0 297161 2666 0 0 23 0 5 0 521666798 4992749568 1141989 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 1218933 1141989 176 36 0 1216838 0
[pid=7090/tid=7097] ppid=7085 vsize=4875732 CPUtime=746.63 cores=0,2,4,6
/proc/7090/task/7097/stat : 7097 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 141875 0 0 0 73759 904 0 0 18 0 5 0 521666806 4992749568 1141989 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=7090/tid=7098] ppid=7085 vsize=4875732 CPUtime=749.15 cores=0,2,4,6
/proc/7090/task/7098/stat : 7098 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 81338 0 0 0 74133 782 0 0 22 0 5 0 521666806 4992749568 1141989 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=7090/tid=7099] ppid=7085 vsize=4875732 CPUtime=754.19 cores=0,2,4,6
/proc/7090/task/7099/stat : 7099 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 807973 0 0 0 75262 157 0 0 25 0 5 0 521666806 4992749568 1141989 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=7090/tid=7100] ppid=7085 vsize=4875732 CPUtime=748.2 cores=0,2,4,6
/proc/7090/task/7100/stat : 7100 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 109464 0 0 0 73999 821 0 0 18 0 5 0 521666806 4992749568 1141989 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2998.27
Current children cumulated vsize (KiB) 4875732

[startup+822.301 s]
/proc/loadavg: 7.95 7.81 7.76 10/186 7122
/proc/meminfo: memFree=9082908/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=5008460 CPUtime=3233.18 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 1181175 0 0 0 320411 2907 0 0 23 0 5 0 521666798 5128663040 1181044 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 1252115 1181044 176 36 0 1250020 0
[pid=7090/tid=7097] ppid=7085 vsize=5008460 CPUtime=804.9 cores=0,2,4,6
/proc/7090/task/7097/stat : 7097 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 143110 0 0 0 79506 984 0 0 25 0 5 0 521666806 5128663040 1181044 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=7090/tid=7098] ppid=7085 vsize=5008460 CPUtime=807.82 cores=0,2,4,6
/proc/7090/task/7098/stat : 7098 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 82989 0 0 0 79926 856 0 0 25 0 5 0 521666806 5128663040 1181044 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=7090/tid=7099] ppid=7085 vsize=5008460 CPUtime=813.36 cores=0,2,4,6
/proc/7090/task/7099/stat : 7099 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 841327 0 0 0 81172 164 0 0 25 0 5 0 521666806 5128663040 1181044 33554432000 4194304 4338100 140735122756960 18446744073709551615 237278274697 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=7090/tid=7100] ppid=7085 vsize=5008460 CPUtime=806.98 cores=0,2,4,6
/proc/7090/task/7100/stat : 7100 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 112279 0 0 0 79798 900 0 0 20 0 5 0 521666806 5128663040 1181044 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3233.18
Current children cumulated vsize (KiB) 5008460

[startup+882.301 s]
/proc/loadavg: 7.98 7.85 7.77 10/186 7124
/proc/meminfo: memFree=8436216/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=5255892 CPUtime=3469.39 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 1242839 0 0 0 343786 3153 0 0 23 0 5 0 521666798 5382033408 1242708 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 1313973 1242708 176 36 0 1311878 0
[pid=7090/tid=7097] ppid=7085 vsize=5255892 CPUtime=863.77 cores=0,2,4,6
/proc/7090/task/7097/stat : 7097 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 155723 0 0 0 85313 1064 0 0 25 0 5 0 521666806 5382033408 1242708 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=7090/tid=7098] ppid=7085 vsize=5255892 CPUtime=866.36 cores=0,2,4,6
/proc/7090/task/7098/stat : 7098 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 85720 0 0 0 85700 936 0 0 25 0 5 0 521666806 5382033408 1242708 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=7090/tid=7099] ppid=7085 vsize=5255892 CPUtime=873.25 cores=0,2,4,6
/proc/7090/task/7099/stat : 7099 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 886568 0 0 0 87152 173 0 0 25 0 5 0 521666806 5382033408 1242708 33554432000 4194304 4338100 140735122756960 18446744073709551615 237278275041 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=7090/tid=7100] ppid=7085 vsize=5255892 CPUtime=865.9 cores=0,2,4,6
/proc/7090/task/7100/stat : 7100 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 113358 0 0 0 85612 978 0 0 25 0 5 0 521666806 5382033408 1242708 33554432000 4194304 4338100 140735122756960 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3469.39
Current children cumulated vsize (KiB) 5255892

[startup+942.3 s]
/proc/loadavg: 8.06 7.88 7.79 7/186 7125
/proc/meminfo: memFree=8074076/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=5473796 CPUtime=3703.81 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 1291285 0 0 0 367003 3378 0 0 23 0 5 0 521666798 5605167104 1291154 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 1368449 1291154 176 36 0 1366354 0
[pid=7090/tid=7097] ppid=7085 vsize=5473796 CPUtime=922.17 cores=0,2,4,6
/proc/7090/task/7097/stat : 7097 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 159729 0 0 0 91082 1135 0 0 25 0 5 0 521666806 5605167104 1291154 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=7090/tid=7098] ppid=7085 vsize=5473796 CPUtime=924.67 cores=0,2,4,6
/proc/7090/task/7098/stat : 7098 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 89009 0 0 0 91457 1010 0 0 18 0 5 0 521666806 5605167104 1291154 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=7090/tid=7099] ppid=7085 vsize=5473796 CPUtime=932.65 cores=0,2,4,6
/proc/7090/task/7099/stat : 7099 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 920610 0 0 0 93085 180 0 0 25 0 5 0 521666806 5605167104 1291154 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=7090/tid=7100] ppid=7085 vsize=5473796 CPUtime=924.19 cores=0,2,4,6
/proc/7090/task/7100/stat : 7100 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 120467 0 0 0 91369 1050 0 0 25 0 5 0 521666806 5605167104 1291154 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3703.81
Current children cumulated vsize (KiB) 5473796

[startup+1002.31 s]
/proc/loadavg: 7.54 7.78 7.76 10/186 7127
/proc/meminfo: memFree=7791192/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=5597140 CPUtime=3936.24 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 1328961 0 0 0 390022 3602 0 0 23 0 5 0 521666798 5731471360 1328830 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 1399285 1328830 176 36 0 1397190 0
[pid=7090/tid=7097] ppid=7085 vsize=5597140 CPUtime=979.86 cores=0,2,4,6
/proc/7090/task/7097/stat : 7097 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 160792 0 0 0 96779 1207 0 0 19 0 5 0 521666806 5731471360 1328830 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=7090/tid=7098] ppid=7085 vsize=5597140 CPUtime=982.5 cores=0,2,4,6
/proc/7090/task/7098/stat : 7098 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 90293 0 0 0 97170 1080 0 0 21 0 5 0 521666806 5731471360 1328830 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=7090/tid=7099] ppid=7085 vsize=5597140 CPUtime=991.57 cores=0,2,4,6
/proc/7090/task/7099/stat : 7099 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 951968 0 0 0 98970 187 0 0 25 0 5 0 521666806 5731471360 1328830 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=7090/tid=7100] ppid=7085 vsize=5597140 CPUtime=982.18 cores=0,2,4,6
/proc/7090/task/7100/stat : 7100 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 124438 0 0 0 97092 1126 0 0 25 0 5 0 521666806 5731471360 1328830 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3936.24
Current children cumulated vsize (KiB) 5597140

[startup+1062.3 s]
/proc/loadavg: 7.97 7.86 7.79 10/186 7129
/proc/meminfo: memFree=7378128/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=5926444 CPUtime=4171.97 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 1374416 0 0 0 413372 3825 0 0 23 0 5 0 521666798 6068678656 1374285 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 1481611 1374285 176 36 0 1479516 0
[pid=7090/tid=7097] ppid=7085 vsize=5926444 CPUtime=1038.63 cores=0,2,4,6
/proc/7090/task/7097/stat : 7097 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 161833 0 0 0 102582 1281 0 0 25 0 5 0 521666806 6068678656 1374285 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=7090/tid=7098] ppid=7085 vsize=5926444 CPUtime=1041.05 cores=0,2,4,6
/proc/7090/task/7098/stat : 7098 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 93501 0 0 0 102956 1149 0 0 25 0 5 0 521666806 6068678656 1374285 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=7090/tid=7099] ppid=7085 vsize=5926444 CPUtime=1051.21 cores=0,2,4,6
/proc/7090/task/7099/stat : 7099 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 990789 0 0 0 104925 196 0 0 25 0 5 0 521666806 6068678656 1374285 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=7090/tid=7100] ppid=7085 vsize=5926444 CPUtime=1040.98 cores=0,2,4,6
/proc/7090/task/7100/stat : 7100 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 126823 0 0 0 102900 1198 0 0 25 0 5 0 521666806 6068678656 1374285 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4171.97
Current children cumulated vsize (KiB) 5926444

[startup+1122.3 s]
/proc/loadavg: 7.84 7.82 7.77 10/186 7130
/proc/meminfo: memFree=6942716/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=6034148 CPUtime=4402.4 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 1415119 0 0 0 436166 4074 0 0 23 0 5 0 521666798 6178967552 1414988 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 1508537 1414988 176 36 0 1506442 0
[pid=7090/tid=7097] ppid=7085 vsize=6034148 CPUtime=1095.44 cores=0,2,4,6
/proc/7090/task/7097/stat : 7097 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 162954 0 0 0 108180 1364 0 0 25 0 5 0 521666806 6178967552 1414988 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=7090/tid=7098] ppid=7085 vsize=6034148 CPUtime=1098.82 cores=0,2,4,6
/proc/7090/task/7098/stat : 7098 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 100720 0 0 0 108656 1226 0 0 19 0 5 0 521666806 6178967552 1414988 33554432000 4194304 4338100 140735122756960 18446744073709551615 4234430 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=7090/tid=7099] ppid=7085 vsize=6034148 CPUtime=1109.11 cores=0,2,4,6
/proc/7090/task/7099/stat : 7099 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 1017345 0 0 0 110708 203 0 0 25 0 5 0 521666806 6178967552 1414988 33554432000 4194304 4338100 140735122756960 18446744073709551615 4284198 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=7090/tid=7100] ppid=7085 vsize=6034148 CPUtime=1098.92 cores=0,2,4,6
/proc/7090/task/7100/stat : 7100 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 132630 0 0 0 108613 1279 0 0 25 0 5 0 521666806 6178967552 1414988 33554432000 4194304 4338100 140735122756960 18446744073709551615 237278274997 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4402.4
Current children cumulated vsize (KiB) 6034148

[startup+1182.3 s]
/proc/loadavg: 7.42 7.68 7.72 7/186 7132
/proc/meminfo: memFree=6668648/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=6204924 CPUtime=4633.62 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 1465354 0 0 0 459010 4352 0 0 23 0 5 0 521666798 6353842176 1465223 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 1551231 1465223 176 36 0 1549136 0
[pid=7090/tid=7097] ppid=7085 vsize=6204924 CPUtime=1152.56 cores=0,2,4,6
/proc/7090/task/7097/stat : 7097 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 164209 0 0 0 113802 1454 0 0 20 0 5 0 521666806 6353842176 1465223 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=7090/tid=7098] ppid=7085 vsize=6204924 CPUtime=1156.41 cores=0,2,4,6
/proc/7090/task/7098/stat : 7098 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 103035 0 0 0 114324 1317 0 0 25 0 5 0 521666806 6353842176 1465223 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=7090/tid=7099] ppid=7085 vsize=6204924 CPUtime=1167.91 cores=0,2,4,6
/proc/7090/task/7099/stat : 7099 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 1060370 0 0 0 116579 212 0 0 25 0 5 0 521666806 6353842176 1465223 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=7090/tid=7100] ppid=7085 vsize=6204924 CPUtime=1156.63 cores=0,2,4,6
/proc/7090/task/7100/stat : 7100 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 136270 0 0 0 114296 1367 0 0 22 0 5 0 521666806 6353842176 1465223 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4633.62
Current children cumulated vsize (KiB) 6204924



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1225.7 s]
/proc/loadavg: 7.20 7.58 7.69 10/186 7133
/proc/meminfo: memFree=6470476/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=6230276 CPUtime=4800.1 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) S 7085 7090 6445 0 -1 4202496 1480766 0 0 0 475502 4508 0 0 23 0 5 0 521666798 6379802624 1480635 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290682341 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 1557569 1480635 176 36 0 1555474 0
[pid=7090/tid=7097] ppid=7085 vsize=6230276 CPUtime=1193.69 cores=0,2,4,6
/proc/7090/task/7097/stat : 7097 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 165079 0 0 0 117867 1502 0 0 25 0 5 0 521666806 6379802624 1480635 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254291 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=7090/tid=7098] ppid=7085 vsize=6230276 CPUtime=1197.93 cores=0,2,4,6
/proc/7090/task/7098/stat : 7098 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 106811 0 0 0 118422 1371 0 0 16 0 5 0 521666806 6379802624 1480635 33554432000 4194304 4338100 140735122756960 18446744073709551615 237290705589 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=7090/tid=7099] ppid=7085 vsize=6230276 CPUtime=1210.13 cores=0,2,4,6
/proc/7090/task/7099/stat : 7099 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 1066904 0 0 0 120797 216 0 0 25 0 5 0 521666806 6379802624 1480635 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254418 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=7090/tid=7100] ppid=7085 vsize=6230276 CPUtime=1198.23 cores=0,2,4,6
/proc/7090/task/7100/stat : 7100 (ttsth-5-4) R 7085 7090 6445 0 -1 4202560 140502 0 0 0 118406 1417 0 0 16 0 5 0 521666806 6379802624 1480635 33554432000 4194304 4338100 140735122756960 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.1
Current children cumulated vsize (KiB) 6230276

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1225.9 s]
/proc/loadavg: 7.20 7.58 7.69 10/186 7133
/proc/meminfo: memFree=6470476/32950904 swapFree=67082284/67111528
[pid=7090] ppid=7085 vsize=0 CPUtime=4800.3 cores=0,2,4,6
/proc/7090/stat : 7090 (ttsth-5-4) Z 7085 7090 6445 0 -1 4203548 1480767 0 0 0 475503 4527 0 0 18 0 2 0 521666798 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/7090/statm: 0 0 0 0 0 0 0
[pid=7090/tid=7099] ppid=7085 vsize=0 CPUtime=1210.32 cores=0,2,4,6
/proc/7090/task/7099/stat : 7099 (ttsth-5-4) R 7085 7090 6445 0 -1 4203588 1066904 0 0 0 120797 235 0 0 25 0 2 0 521666806 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.3
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1225.92
CPU time (s): 4800.33
CPU user time (s): 4755.04
CPU system time (s): 45.2921
CPU usage (%): 391.57
Max. virtual memory (cumulated for all children) (KiB): 6230276

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4755.04
system time used= 45.2921
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1480767
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 373978
involuntary context switches= 28488

runsolver used 2.13368 second user time and 4.23735 second system time

The end

Launcher Data 

Begin job on node121 at 2011-04-29 20:11:10
IDJOB=3264950
IDBENCH=85362
IDSOLVER=1654
FILE ID=node121/3264950-1304100670
RUNJOBID= node121-1304100670-7071
PBS_JOBID= 13170609
Free space on /tmp= 72656 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT05/crafted/jarvisalo05/mod2c-rand3bip-sat/mod2c-rand3bip-sat-240-2.shuffled-as.sat05-2519.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3264950-1304100670/watcher-3264950-1304100670 -o /tmp/evaluation-result-3264950-1304100670/solver-3264950-1304100670 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3264950-1304100670.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= b6a88c8ae73b20cfc873207d1a0caa09
RANDOM SEED=2020951381

node121.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950904 kB
MemFree:      19936776 kB
Buffers:       2183812 kB
Cached:       10272408 kB
SwapCached:       2776 kB
Active:        9125188 kB
Inactive:      3369888 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950904 kB
LowFree:      19936776 kB
SwapTotal:    67111528 kB
SwapFree:     67082284 kB
Dirty:              44 kB
Writeback:           0 kB
AnonPages:       35772 kB
Mapped:          13532 kB
Slab:           455924 kB
PageTables:       4720 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586980 kB
Committed_AS:   188488 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72652 MiB
End job on node121 at 2011-04-29 20:31:38