Trace number 3264994

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.18 1215.26

General information on the benchmark

NameSAT05/crafted/jarvisalo05/mod2c-rand3bip-sat/
mod2c-rand3bip-sat-230-3.shuffled-as.sat05-2505.cnf
MD5SUMb9f02ee542f99ae0fd8f291dd570c53a
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark18.2832
Satisfiable
(Un)Satisfiability was proved
Number of variables326
Number of clauses2332
Sum of the clauses size10392
Maximum clause length6
Minimum clause length4
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 41416
Number of clauses of size 5768
Number of clauses of size over 5148

Solver Data 

0.00/0.00	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3264994-1304101229.cnf 
0.00/0.00	c Using SATTIMEOUT of 4800
0.00/0.00	c Using SATRAM of 15500
0.00/0.00	c Size of subset node = 32
0.00/0.00	c Variables = 326
0.00/0.00	c Clauses = 2332
0.00/0.00	c Literals = 10392
0.00/0.00	c Reduced size:
0.00/0.00	c Variables = 326
0.00/0.00	c Clauses = 2332
0.00/0.00	c Literals = 10392
0.00/0.00	c a2
0.00/0.00	c a1
0.01/0.07	c Independent sub-problems = 1
0.01/0.07	c Sub-problem with 326 variables and 2332 clauses
0.01/0.07	c using permutation method 4
0.01/0.07	c count = 2
0.01/0.07	c Ordering finished 0.070 seconds
0.01/0.07	c mapped 0.070 seconds
0.01/0.07	c permuted 0.070 seconds
0.01/0.07	c about to sort
0.01/0.07	c sorted
0.01/0.08	c clauses now 2332
0.01/0.08	c init finished 0.070 seconds
0.01/0.08	c tree built 0.070 seconds
0.01/0.08	c 
0.01/0.08	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.01/0.08	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.01/0.08	c ................................ | ..........                    |                                  0.1         0.0             
0.01/0.08	c ................................ | ..........                  1 |                                  0.1        10.0             
0.01/0.08	c ................................ | ..........                  2 |                                  0.1        20.0             
0.01/0.08	c ................................ | ..........                  3 |                                  0.1        30.0             
0.01/0.08	c ................................ | ..........                  4 |                                  0.1        40.0             
0.01/0.08	c ................................ | ..........                  5 |                                  0.1        50.0             
0.01/0.08	c ................................ | ..........               1000 | 70           148                 0.1     10000.0         1339
95.87/24.15	c ................................ | ..........            1206000 | 326881       447689             96.0     12556.0     14368220
191.87/48.25	c ................................ | ..........            2025000 | 558394       736864            192.1     10541.4     22522797
287.85/72.35	c ................................ | ..........            2716000 | 761130       957366            288.1      9427.9     28491101
383.80/96.45	c ................................ | ..........            3444000 | 967031       1197083           384.0      8968.0     32244876
479.67/120.59	c ................................ | *.........            4220000 | 1186784      1457024           480.0      8791.1     38697763
576.00/144.71	c ................................ | *.........            4895000 | 1383463      1681798           576.1      8497.1     44345812
671.97/168.82	c ................................ | *.........            5509000 | 1564802      1873398           672.1      8196.9     49716126
768.08/193.00	c ................................ | *.........            6196000 | 1765292      2102646           768.0      8067.4     53099200
863.94/217.23	c ................................ | *.........            6875000 | 1962702      2334647           864.0      7956.9     56668120
960.14/241.40	c ................................ | **........            7460000 | 2138317      2517533           960.0      7770.4     59204633
1056.00/265.50	c ................................ | **........            8109000 | 2328467      2733786          1056.0      7678.7     64091507
1151.74/289.77	c ................................ | **........            8768000 | 2518682      2963771          1152.0      7610.9     67350694
1247.73/313.99	c ................................ | **........            9324000 | 2686220      3145444          1248.1      7470.4     71294823
1343.85/338.15	c ................................ | **........            9842000 | 2840979      3308518          1344.1      7322.5     73547061
1439.90/362.35	c ................................ | ***.......           10374000 | 2994289      3472789          1440.1      7203.7     74183286
1536.09/386.52	c ................................ | ***.......           10895000 | 3144319      3636714          1536.1      7092.5     74852985
1631.97/410.76	c ................................ | ***.......           11367000 | 3282880      3777825          1632.2      6964.1     76425091
1727.87/434.99	c ................................ | ***.......           11801000 | 3410519      3903259          1728.2      6828.4     76948344
1824.00/459.22	c ................................ | ***.......           12221000 | 3534294      4034031          1824.1      6699.7     76948344
1919.94/483.55	c ................................ | ****......           12769000 | 3693250      4212588          1920.2      6650.0     76948344
2016.00/507.80	c ................................ | ****......           13303000 | 3847886      4385685          2016.0      6598.7     77333610
2111.79/531.96	c ................................ | ****......           13757000 | 3982474      4526530          2112.1      6513.5     78202864
2208.18/556.41	c ................................ | ****......           14217000 | 4118024      4666371          2208.2      6438.2     79044214
2303.85/580.64	c ................................ | ****......           14727000 | 4266412      4834014          2304.0      6391.9     80146562
2400.05/605.01	c ................................ | *****.....           15231000 | 4413029      5002126          2400.1      6346.0     81224316
2495.71/629.37	c ................................ | *****.....           15696000 | 4551600      5148543          2496.0      6288.4     83900930
2592.01/653.72	c ................................ | *****.....           16136000 | 4683459      5287318          2592.1      6225.1     85415720
2687.84/678.08	c ................................ | *****.....           16606000 | 4820863      5442717          2688.2      6177.4     87676245
2783.90/702.49	c ................................ | *****.....           17103000 | 4965303      5612107          2784.2      6142.9     88756383
2879.75/726.86	c ................................ | ******....           17567000 | 5103877      5763711          2880.0      6099.7     91202341
2975.73/751.08	c ................................ | ******....           17989000 | 5231828      5897365          2976.1      6044.5     93342667
3071.76/775.37	c ................................ | ******....           18402000 | 5357628      6031954          3072.1      5990.1     96055378
3167.88/799.77	c ................................ | ******....           18849000 | 5491717      6193121          3168.2      5949.5     96725601
3263.83/824.25	c ................................ | ******....           19278000 | 5619651      6352277          3264.0      5906.2     97979587
3359.98/848.74	c ................................ | *******...           19696000 | 5746522      6503349          3360.1      5861.7     97979587
3456.03/873.02	c ................................ | *******...           20110000 | 5870460      6657528          3456.1      5818.6     98981149
3551.77/897.19	c ................................ | *******...           20518000 | 5992382      6812036          3552.2      5776.2    100624705
3647.72/921.70	c ................................ | *******...           20956000 | 6123274      6972738          3648.0      5744.5    103638634
3743.96/946.05	c ................................ | *******...           21409000 | 6259388      7136268          3744.0      5718.2    107576781
3839.85/970.59	c ................................ | ********..           21830000 | 6385699      7288437          3840.2      5684.6    109111714
3935.96/994.92	c ................................ | ********..           22263000 | 6515728      7446544          3936.1      5656.2    111699996
4031.85/1019.38	c ................................ | ********..           22693000 | 6644795      7607695          4032.2      5628.0    112579452
4128.04/1043.86	c ................................ | ********..           23112000 | 6769798      7767310          4128.3      5598.5    113235781
4224.06/1068.22	c ................................ | ********..           23553000 | 6900387      7938166          4224.2      5575.8    114515543
4320.08/1092.80	c ................................ | *********.           23968000 | 7024655      8085369          4320.1      5548.0    114993090
4415.80/1117.09	c ................................ | *********.           24388000 | 7149092      8242998          4416.2      5522.4    114993090
4512.03/1141.61	c ................................ | *********.           24796000 | 7273492      8390093          4512.1      5495.5    115346308
4608.10/1166.02	c ................................ | *********.           25208000 | 7398326      8533544          4608.2      5470.3    116141531
4703.77/1190.49	c ................................ | *********.           25609000 | 7517359      8679407          4704.2      5443.9    117360319
4799.63/1215.09	c CPU limit (4800) reached
4799.63/1215.09	c Time = 4800.01 seconds
4799.63/1215.09	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3264994-1304101229/watcher-3264994-1304101229 -o /tmp/evaluation-result-3264994-1304101229/solver-3264994-1304101229 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3264994-1304101229.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.20 7.89 7.95 2/174 18256
/proc/meminfo: memFree=26301324/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=9640 CPUtime=0 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) R 18254 18256 17388 0 -1 4202496 480 0 0 0 0 0 0 0 24 0 1 0 521722614 9871360 421 33554432000 4194304 4338100 140737404215296 18446744073709551615 4206980 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/18256/statm: 2410 421 152 36 0 315 0

[startup+0.014762 s]
/proc/loadavg: 7.20 7.89 7.95 2/174 18256
/proc/meminfo: memFree=26301324/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=9640 CPUtime=0.01 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) R 18254 18256 17388 0 -1 4202496 492 0 0 0 1 0 0 0 24 0 1 0 521722614 9871360 433 33554432000 4194304 4338100 140737404215296 18446744073709551615 4295932 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/18256/statm: 2410 433 155 36 0 315 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 9640

[startup+0.108656 s]
/proc/loadavg: 7.20 7.89 7.95 2/174 18256
/proc/meminfo: memFree=26301324/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=198104 CPUtime=0.17 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 1998 0 0 0 17 0 0 0 24 0 5 0 521722614 202858496 1904 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 49526 1905 175 36 0 47431 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 198104

[startup+0.300711 s]
/proc/loadavg: 7.20 7.89 7.95 2/174 18256
/proc/meminfo: memFree=26301324/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=204756 CPUtime=0.94 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 4525 0 0 0 93 1 0 0 24 0 5 0 521722614 209670144 4431 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 51189 4433 175 36 0 49094 0
Current children cumulated CPU time (s) 0.94
Current children cumulated vsize (KiB) 204756

[startup+0.700635 s]
/proc/loadavg: 7.20 7.89 7.95 2/174 18256
/proc/meminfo: memFree=26301324/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=217528 CPUtime=2.53 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 8043 0 0 0 251 2 0 0 24 0 5 0 521722614 222748672 7949 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 54382 7949 175 36 0 52287 0
Current children cumulated CPU time (s) 2.53
Current children cumulated vsize (KiB) 217528

[startup+1.50048 s]
/proc/loadavg: 7.20 7.89 7.95 9/186 18271
/proc/meminfo: memFree=26202772/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=305344 CPUtime=5.73 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 14564 0 0 0 568 5 0 0 24 0 5 0 521722614 312672256 14470 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 76336 14470 175 36 0 74241 0
[pid=18256/tid=18264] ppid=18254 vsize=305344 CPUtime=1.41 cores=0,2,4,6
/proc/18256/task/18264/stat : 18264 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 2856 0 0 0 140 1 0 0 18 0 5 0 521722623 312672256 14470 33554432000 4194304 4338100 140737404215296 18446744073709551615 208079627494 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18256/tid=18265] ppid=18254 vsize=305344 CPUtime=1.41 cores=0,2,4,6
/proc/18256/task/18265/stat : 18265 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 4075 0 0 0 140 1 0 0 20 0 5 0 521722623 312672256 14470 33554432000 4194304 4338100 140737404215296 18446744073709551615 4242646 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=305344 CPUtime=1.4 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 3753 0 0 0 139 1 0 0 25 0 5 0 521722623 312672256 14470 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18256/tid=18267] ppid=18254 vsize=305344 CPUtime=1.41 cores=0,2,4,6
/proc/18256/task/18267/stat : 18267 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 2452 0 0 0 140 1 0 0 20 0 5 0 521722623 312672256 14471 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254221 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.73
Current children cumulated vsize (KiB) 305344

[startup+3.10118 s]
/proc/loadavg: 7.20 7.89 7.95 9/186 18271
/proc/meminfo: memFree=26122676/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=352776 CPUtime=12.12 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 26101 0 0 0 1202 10 0 0 24 0 5 0 521722614 361242624 26007 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 88194 26007 175 36 0 86099 0
[pid=18256/tid=18264] ppid=18254 vsize=352776 CPUtime=3 cores=0,2,4,6
/proc/18256/task/18264/stat : 18264 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 5497 0 0 0 298 2 0 0 19 0 5 0 521722623 361242624 26007 33554432000 4194304 4338100 140737404215296 18446744073709551615 208079928880 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18256/tid=18265] ppid=18254 vsize=352776 CPUtime=3 cores=0,2,4,6
/proc/18256/task/18265/stat : 18265 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 7615 0 0 0 299 1 0 0 25 0 5 0 521722623 361242624 26007 33554432000 4194304 4338100 140737404215296 18446744073709551615 208079928880 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=352776 CPUtime=3 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 6920 0 0 0 297 3 0 0 25 0 5 0 521722623 361242624 26007 33554432000 4194304 4338100 140737404215296 18446744073709551615 208079628002 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18256/tid=18267] ppid=18254 vsize=352776 CPUtime=3.01 cores=0,2,4,6
/proc/18256/task/18267/stat : 18267 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 4641 0 0 0 299 2 0 0 19 0 5 0 521722623 361242624 26007 33554432000 4194304 4338100 140737404215296 18446744073709551615 208079901039 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 12.12
Current children cumulated vsize (KiB) 352776

[startup+6.30057 s]
/proc/loadavg: 7.26 7.89 7.95 9/186 18271
/proc/meminfo: memFree=25940580/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=415720 CPUtime=24.88 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 42152 0 0 0 2468 20 0 0 24 0 5 0 521722614 425697280 42058 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 103930 42058 175 36 0 101835 0
[pid=18256/tid=18264] ppid=18254 vsize=415720 CPUtime=6.2 cores=0,2,4,6
/proc/18256/task/18264/stat : 18264 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 9364 0 0 0 614 6 0 0 25 0 5 0 521722623 425697280 42058 33554432000 4194304 4338100 140737404215296 18446744073709551615 208079627513 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18256/tid=18265] ppid=18254 vsize=415720 CPUtime=6.19 cores=0,2,4,6
/proc/18256/task/18265/stat : 18265 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 12144 0 0 0 617 2 0 0 18 0 5 0 521722623 425697280 42059 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092058293 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=415720 CPUtime=6.17 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 11106 0 0 0 610 7 0 0 25 0 5 0 521722623 425697280 42059 33554432000 4194304 4338100 140737404215296 18446744073709551615 208079627489 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18256/tid=18267] ppid=18254 vsize=415720 CPUtime=6.2 cores=0,2,4,6
/proc/18256/task/18267/stat : 18267 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 8110 0 0 0 617 3 0 0 25 0 5 0 521722623 425697280 42059 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092058293 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 24.88
Current children cumulated vsize (KiB) 415720

[startup+12.7004 s]
/proc/loadavg: 7.32 7.90 7.95 9/186 18271
/proc/meminfo: memFree=25670656/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=513820 CPUtime=50.4 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 66920 0 0 0 4996 44 0 0 24 0 5 0 521722614 526151680 66826 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 128455 66826 176 36 0 126360 0
[pid=18256/tid=18264] ppid=18254 vsize=513820 CPUtime=12.59 cores=0,2,4,6
/proc/18256/task/18264/stat : 18264 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 15474 0 0 0 1245 14 0 0 25 0 5 0 521722623 526151680 66826 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254239 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18256/tid=18265] ppid=18254 vsize=513820 CPUtime=12.57 cores=0,2,4,6
/proc/18256/task/18265/stat : 18265 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 18892 0 0 0 1252 5 0 0 18 0 5 0 521722623 526151680 66826 33554432000 4194304 4338100 140737404215296 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=513820 CPUtime=12.53 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 17542 0 0 0 1238 15 0 0 25 0 5 0 521722623 526151680 66826 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254337 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18256/tid=18267] ppid=18254 vsize=513820 CPUtime=12.6 cores=0,2,4,6
/proc/18256/task/18267/stat : 18267 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 13584 0 0 0 1251 9 0 0 25 0 5 0 521722623 526151680 66826 33554432000 4194304 4338100 140737404215296 18446744073709551615 208079928752 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 50.4
Current children cumulated vsize (KiB) 513820

[startup+25.5009 s]
/proc/loadavg: 7.47 7.90 7.95 9/186 18271
/proc/meminfo: memFree=25147756/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=726684 CPUtime=101.43 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 122331 0 0 0 10055 88 0 0 24 0 5 0 521722614 744124416 122237 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 181671 122237 176 36 0 179576 0
[pid=18256/tid=18264] ppid=18254 vsize=726684 CPUtime=25.38 cores=0,2,4,6
/proc/18256/task/18264/stat : 18264 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 29055 0 0 0 2511 27 0 0 25 0 5 0 521722623 744124416 122237 33554432000 4194304 4338100 140737404215296 18446744073709551615 208079890235 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18256/tid=18265] ppid=18254 vsize=726684 CPUtime=25.3 cores=0,2,4,6
/proc/18256/task/18265/stat : 18265 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 33679 0 0 0 2520 10 0 0 18 0 5 0 521722623 744124416 122237 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092058293 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=726684 CPUtime=25.25 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 31498 0 0 0 2494 31 0 0 25 0 5 0 521722623 744124416 122237 33554432000 4194304 4338100 140737404215296 18446744073709551615 4279542 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18256/tid=18267] ppid=18254 vsize=726684 CPUtime=25.38 cores=0,2,4,6
/proc/18256/task/18267/stat : 18267 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 26671 0 0 0 2520 18 0 0 25 0 5 0 521722623 744124416 122237 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092058293 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 101.43
Current children cumulated vsize (KiB) 726684

[startup+51.101 s]
/proc/loadavg: 7.65 7.91 7.95 9/186 18272
/proc/meminfo: memFree=24472188/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=993980 CPUtime=203.42 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 189309 0 0 0 20181 161 0 0 24 0 5 0 521722614 1017835520 189215 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 248495 189215 176 36 0 246400 0
[pid=18256/tid=18264] ppid=18254 vsize=993980 CPUtime=50.83 cores=0,2,4,6
/proc/18256/task/18264/stat : 18264 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 47297 0 0 0 5029 54 0 0 25 0 5 0 521722623 1017835520 189215 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18256/tid=18265] ppid=18254 vsize=993980 CPUtime=50.8 cores=0,2,4,6
/proc/18256/task/18265/stat : 18265 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 48619 0 0 0 5064 16 0 0 20 0 5 0 521722623 1017835520 189215 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092058293 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=993980 CPUtime=50.73 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 48710 0 0 0 5018 55 0 0 25 0 5 0 521722623 1017835520 189215 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254429 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18256/tid=18267] ppid=18254 vsize=993980 CPUtime=50.94 cores=0,2,4,6
/proc/18256/task/18267/stat : 18267 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 43255 0 0 0 5060 34 0 0 25 0 5 0 521722623 1017835520 189215 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092058293 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 203.42
Current children cumulated vsize (KiB) 993980

[startup+102.306 s]
/proc/loadavg: 7.89 7.94 7.96 9/186 18273
/proc/meminfo: memFree=23465144/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=1359500 CPUtime=407.33 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 282869 0 0 0 40450 283 0 0 24 0 5 0 521722614 1392128000 282775 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 339875 282775 176 36 0 337780 0
[pid=18256/tid=18264] ppid=18254 vsize=1359500 CPUtime=101.6 cores=0,2,4,6
/proc/18256/task/18264/stat : 18264 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 75482 0 0 0 10063 97 0 0 25 0 5 0 521722623 1392128000 282776 33554432000 4194304 4338100 140737404215296 18446744073709551615 208079627373 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18256/tid=18265] ppid=18254 vsize=1359500 CPUtime=101.79 cores=0,2,4,6
/proc/18256/task/18265/stat : 18265 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 72894 0 0 0 10152 27 0 0 18 0 5 0 521722623 1392128000 282776 33554432000 4194304 4338100 140737404215296 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=1359500 CPUtime=101.79 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 73680 0 0 0 10081 98 0 0 25 0 5 0 521722623 1392128000 282777 33554432000 4194304 4338100 140737404215296 18446744073709551615 4206902 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18256/tid=18267] ppid=18254 vsize=1359500 CPUtime=102.04 cores=0,2,4,6
/proc/18256/task/18267/stat : 18267 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 59387 0 0 0 10144 60 0 0 18 0 5 0 521722623 1392128000 282778 33554432000 4194304 4338100 140737404215296 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 407.33
Current children cumulated vsize (KiB) 1359500

[startup+162.301 s]
/proc/loadavg: 8.00 7.96 7.96 9/186 18275
/proc/meminfo: memFree=22500268/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=1811356 CPUtime=646.13 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 398089 0 0 0 64184 429 0 0 24 0 5 0 521722614 1854828544 397995 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 452839 397995 176 36 0 450744 0
[pid=18256/tid=18264] ppid=18254 vsize=1811356 CPUtime=161.23 cores=0,2,4,6
/proc/18256/task/18264/stat : 18264 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 111461 0 0 0 15983 140 0 0 25 0 5 0 521722623 1854828544 397995 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092058293 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=18256/tid=18265] ppid=18254 vsize=1811356 CPUtime=161.58 cores=0,2,4,6
/proc/18256/task/18265/stat : 18265 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 98168 0 0 0 16100 58 0 0 25 0 5 0 521722623 1854828544 397995 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=1811356 CPUtime=161.49 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 103810 0 0 0 16021 128 0 0 18 0 5 0 521722623 1854828544 397995 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092058293 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18256/tid=18267] ppid=18254 vsize=1811356 CPUtime=161.72 cores=0,2,4,6
/proc/18256/task/18267/stat : 18267 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 83222 0 0 0 16071 101 0 0 25 0 5 0 521722623 1854828544 397995 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 646.13
Current children cumulated vsize (KiB) 1811356


################
# More data... #
################

/proc/18256/task/18267/stat : 18267 (ttsth-5-4) S 18254 18256 17388 0 -1 4202560 138376 0 0 0 69432 263 0 0 25 0 5 0 521722623 3225022464 748325 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092045062 0 0 4096 0 18446604453861128192 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2783.58
Current children cumulated vsize (KiB) 3149436

[startup+762.307 s]
/proc/loadavg: 8.03 8.01 7.99 9/186 18291
/proc/meminfo: memFree=18764248/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=3367444 CPUtime=3020.58 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 805060 0 0 0 300578 1480 0 0 24 0 5 0 521722614 3448262656 804966 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 841861 804966 176 36 0 839766 0
[pid=18256/tid=18264] ppid=18254 vsize=3367444 CPUtime=753.08 cores=0,2,4,6
/proc/18256/task/18264/stat : 18264 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 294505 0 0 0 74807 501 0 0 25 0 5 0 521722623 3448262656 804966 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18256/tid=18265] ppid=18254 vsize=3367444 CPUtime=755.96 cores=0,2,4,6
/proc/18256/task/18265/stat : 18265 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 149301 0 0 0 75304 292 0 0 18 0 5 0 521722623 3448262656 804966 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092058293 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=3367444 CPUtime=754.82 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 210289 0 0 0 75078 404 0 0 25 0 5 0 521722623 3448262656 804966 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18256/tid=18267] ppid=18254 vsize=3367444 CPUtime=756.61 cores=0,2,4,6
/proc/18256/task/18267/stat : 18267 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 149537 0 0 0 75379 282 0 0 25 0 5 0 521722623 3448262656 804966 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3020.58
Current children cumulated vsize (KiB) 3367444

[startup+822.301 s]
/proc/loadavg: 7.94 7.98 7.98 9/186 18292
/proc/meminfo: memFree=18362828/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=3441692 CPUtime=3256.64 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 825493 0 0 0 324099 1565 0 0 24 0 5 0 521722614 3524292608 825399 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 860423 825399 176 36 0 858328 0
[pid=18256/tid=18264] ppid=18254 vsize=3441692 CPUtime=811.68 cores=0,2,4,6
/proc/18256/task/18264/stat : 18264 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 299897 0 0 0 80636 532 0 0 25 0 5 0 521722623 3524292608 825399 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18256/tid=18265] ppid=18254 vsize=3441692 CPUtime=814.93 cores=0,2,4,6
/proc/18256/task/18265/stat : 18265 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 157539 0 0 0 81195 298 0 0 18 0 5 0 521722623 3524292608 825399 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=3441692 CPUtime=813.87 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 213870 0 0 0 80952 435 0 0 25 0 5 0 521722623 3524292608 825400 33554432000 4194304 4338100 140737404215296 18446744073709551615 208079627401 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18256/tid=18267] ppid=18254 vsize=3441692 CPUtime=816.05 cores=0,2,4,6
/proc/18256/task/18267/stat : 18267 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 152760 0 0 0 81307 298 0 0 25 0 5 0 521722623 3524292608 825401 33554432000 4194304 4338100 140737404215296 18446744073709551615 208079928777 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3256.64
Current children cumulated vsize (KiB) 3441692

[startup+882.301 s]
/proc/loadavg: 7.98 7.98 7.98 9/186 18294
/proc/meminfo: memFree=18052176/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=3484660 CPUtime=3492.82 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 838702 0 0 0 347633 1649 0 0 24 0 5 0 521722614 3568291840 838608 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 871165 838608 176 36 0 869070 0
[pid=18256/tid=18264] ppid=18254 vsize=3484660 CPUtime=870.4 cores=0,2,4,6
/proc/18256/task/18264/stat : 18264 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 308297 0 0 0 86477 563 0 0 20 0 5 0 521722623 3568291840 838608 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092058293 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18256/tid=18265] ppid=18254 vsize=3484660 CPUtime=874.21 cores=0,2,4,6
/proc/18256/task/18265/stat : 18265 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 158115 0 0 0 87116 305 0 0 25 0 5 0 521722623 3568291840 838608 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=3484660 CPUtime=872.72 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 216790 0 0 0 86807 465 0 0 25 0 5 0 521722623 3568291840 838608 33554432000 4194304 4338100 140737404215296 18446744073709551615 4240537 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=18256/tid=18267] ppid=18254 vsize=3484660 CPUtime=875.37 cores=0,2,4,6
/proc/18256/task/18267/stat : 18267 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 154072 0 0 0 87223 314 0 0 25 0 5 0 521722623 3568291840 838608 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3492.82
Current children cumulated vsize (KiB) 3484660

[startup+942.3 s]
/proc/loadavg: 7.99 7.98 7.98 9/186 18296
/proc/meminfo: memFree=17710156/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=3721820 CPUtime=3729.28 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 900138 0 0 0 371180 1748 0 0 24 0 5 0 521722614 3811143680 900044 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 930455 900044 176 36 0 928360 0
[pid=18256/tid=18264] ppid=18254 vsize=3721820 CPUtime=929.15 cores=0,2,4,6
/proc/18256/task/18264/stat : 18264 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 336564 0 0 0 92339 576 0 0 18 0 5 0 521722623 3811143680 900044 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092058293 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18256/tid=18265] ppid=18254 vsize=3721820 CPUtime=933.59 cores=0,2,4,6
/proc/18256/task/18265/stat : 18265 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 169226 0 0 0 93018 341 0 0 25 0 5 0 521722623 3811143680 900044 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=3721820 CPUtime=932.09 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 231261 0 0 0 92723 486 0 0 25 0 5 0 521722623 3811143680 900044 33554432000 4194304 4338100 140737404215296 18446744073709551615 208079928800 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=18256/tid=18267] ppid=18254 vsize=3721820 CPUtime=934.33 cores=0,2,4,6
/proc/18256/task/18267/stat : 18267 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 161659 0 0 0 93090 343 0 0 25 0 5 0 521722623 3811143680 900044 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3729.28
Current children cumulated vsize (KiB) 3721820

[startup+1002.3 s]
/proc/loadavg: 8.01 7.98 7.98 9/186 18297
/proc/meminfo: memFree=17379052/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=3889908 CPUtime=3965.19 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 944010 0 0 0 394672 1847 0 0 24 0 5 0 521722614 3983265792 943916 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 972477 943916 176 36 0 970382 0
[pid=18256/tid=18264] ppid=18254 vsize=3889908 CPUtime=987.82 cores=0,2,4,6
/proc/18256/task/18264/stat : 18264 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 359536 0 0 0 98194 588 0 0 19 0 5 0 521722623 3983265792 943916 33554432000 4194304 4338100 140737404215296 18446744073709551615 4287250 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18256/tid=18265] ppid=18254 vsize=3889908 CPUtime=992.78 cores=0,2,4,6
/proc/18256/task/18265/stat : 18265 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 171905 0 0 0 98904 374 0 0 25 0 5 0 521722623 3983265792 943917 33554432000 4194304 4338100 140737404215296 18446744073709551615 4255963 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=3889908 CPUtime=991 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 244771 0 0 0 98592 508 0 0 25 0 5 0 521722623 3983265792 943917 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092058293 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=18256/tid=18267] ppid=18254 vsize=3889908 CPUtime=993.47 cores=0,2,4,6
/proc/18256/task/18267/stat : 18267 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 166370 0 0 0 98972 375 0 0 25 0 5 0 521722623 3983265792 943917 33554432000 4194304 4338100 140737404215296 18446744073709551615 4283573 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3965.19
Current children cumulated vsize (KiB) 3889908

[startup+1062.3 s]
/proc/loadavg: 8.04 8.00 7.98 8/186 18299
/proc/meminfo: memFree=17234692/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=3961412 CPUtime=4200.67 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 964102 0 0 0 418136 1931 0 0 24 0 5 0 521722614 4056485888 964008 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 990353 964008 176 36 0 988258 0
[pid=18256/tid=18264] ppid=18254 vsize=3961412 CPUtime=1046.44 cores=0,2,4,6
/proc/18256/task/18264/stat : 18264 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 372115 0 0 0 104034 610 0 0 25 0 5 0 521722623 4056485888 964009 33554432000 4194304 4338100 140737404215296 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18256/tid=18265] ppid=18254 vsize=3961412 CPUtime=1051.72 cores=0,2,4,6
/proc/18256/task/18265/stat : 18265 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 172874 0 0 0 104783 389 0 0 18 0 5 0 521722623 4056485888 964009 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=3961412 CPUtime=1049.7 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 250700 0 0 0 104438 532 0 0 25 0 5 0 521722623 4056485888 964009 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18256/tid=18267] ppid=18254 vsize=3961412 CPUtime=1052.71 cores=0,2,4,6
/proc/18256/task/18267/stat : 18267 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 166985 0 0 0 104872 399 0 0 25 0 5 0 521722623 4056485888 964009 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092058293 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4200.67
Current children cumulated vsize (KiB) 3961412

[startup+1122.3 s]
/proc/loadavg: 7.97 7.99 7.98 9/186 18300
/proc/meminfo: memFree=17162892/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=4016332 CPUtime=4436.59 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 975316 0 0 0 441649 2010 0 0 24 0 5 0 521722614 4112723968 975222 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 1004083 975222 176 36 0 1001988 0
[pid=18256/tid=18264] ppid=18254 vsize=4016332 CPUtime=1105.1 cores=0,2,4,6
/proc/18256/task/18264/stat : 18264 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 379130 0 0 0 109870 640 0 0 25 0 5 0 521722623 4112723968 975222 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18256/tid=18265] ppid=18254 vsize=4016332 CPUtime=1110.86 cores=0,2,4,6
/proc/18256/task/18265/stat : 18265 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 173979 0 0 0 110694 392 0 0 20 0 5 0 521722623 4112723968 975222 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=4016332 CPUtime=1108.64 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 253225 0 0 0 110301 563 0 0 25 0 5 0 521722623 4112723968 975222 33554432000 4194304 4338100 140737404215296 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18256/tid=18267] ppid=18254 vsize=4016332 CPUtime=1111.88 cores=0,2,4,6
/proc/18256/task/18267/stat : 18267 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 167554 0 0 0 110774 414 0 0 25 0 5 0 521722623 4112723968 975222 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092058293 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4436.59
Current children cumulated vsize (KiB) 4016332

[startup+1182.3 s]
/proc/loadavg: 8.06 8.02 7.99 9/186 18302
/proc/meminfo: memFree=17022508/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=4080940 CPUtime=4671.85 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) S 18254 18256 17388 0 -1 4202496 993442 0 0 0 465098 2087 0 0 24 0 5 0 521722614 4178882560 993348 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092035045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 1020235 993348 176 36 0 1018140 0
[pid=18256/tid=18264] ppid=18254 vsize=4080940 CPUtime=1163.54 cores=0,2,4,6
/proc/18256/task/18264/stat : 18264 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 394583 0 0 0 115678 676 0 0 25 0 5 0 521722623 4178882560 993348 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092058293 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18256/tid=18265] ppid=18254 vsize=4080940 CPUtime=1169.76 cores=0,2,4,6
/proc/18256/task/18265/stat : 18265 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 175568 0 0 0 116581 395 0 0 18 0 5 0 521722623 4178882560 993348 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092058293 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=4080940 CPUtime=1167.51 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 253781 0 0 0 116163 588 0 0 21 0 5 0 521722623 4178882560 993348 33554432000 4194304 4338100 140737404215296 18446744073709551615 4289298 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18256/tid=18267] ppid=18254 vsize=4080940 CPUtime=1170.93 cores=0,2,4,6
/proc/18256/task/18267/stat : 18267 (ttsth-5-4) R 18254 18256 17388 0 -1 4202560 168082 0 0 0 116667 426 0 0 25 0 5 0 521722623 4178882560 993348 33554432000 4194304 4338100 140737404215296 18446744073709551615 208092058293 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4671.85
Current children cumulated vsize (KiB) 4080940



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1215.1 s]
/proc/loadavg: 7.57 7.91 7.96 2/174 18321
/proc/meminfo: memFree=22360004/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=0 CPUtime=4800.02 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) Z 18254 18256 17388 0 -1 4203548 1006135 0 0 0 477870 2132 0 0 18 0 2 0 521722614 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 0 0 0 0 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=0 CPUtime=1199.54 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4203588 254102 0 0 0 119349 605 0 0 25 0 2 0 521722623 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.02
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1215.21 s]
/proc/loadavg: 7.57 7.91 7.96 2/174 18321
/proc/meminfo: memFree=22360004/32950928 swapFree=67111364/67111528
[pid=18256] ppid=18254 vsize=0 CPUtime=4800.13 cores=0,2,4,6
/proc/18256/stat : 18256 (ttsth-5-4) Z 18254 18256 17388 0 -1 4203548 1006135 0 0 0 477870 2143 0 0 18 0 2 0 521722614 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18256/statm: 0 0 0 0 0 0 0
[pid=18256/tid=18266] ppid=18254 vsize=0 CPUtime=1199.64 cores=0,2,4,6
/proc/18256/task/18266/stat : 18266 (ttsth-5-4) R 18254 18256 17388 0 -1 4203588 254102 0 0 0 119349 615 0 0 25 0 2 0 521722623 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.13
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1215.26
CPU time (s): 4800.18
CPU user time (s): 4778.71
CPU system time (s): 21.4767
CPU usage (%): 394.993
Max. virtual memory (cumulated for all children) (KiB): 4125884

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4778.71
system time used= 21.4767
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1006135
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 77402
involuntary context switches= 20754

runsolver used 1.79773 second user time and 3.9704 second system time

The end

Launcher Data 

Begin job on node127 at 2011-04-29 20:20:29
IDJOB=3264994
IDBENCH=85366
IDSOLVER=1654
FILE ID=node127/3264994-1304101229
RUNJOBID= node127-1304101229-18242
PBS_JOBID= 13170587
Free space on /tmp= 71820 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT05/crafted/jarvisalo05/mod2c-rand3bip-sat/mod2c-rand3bip-sat-230-3.shuffled-as.sat05-2505.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3264994-1304101229/watcher-3264994-1304101229 -o /tmp/evaluation-result-3264994-1304101229/solver-3264994-1304101229 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3264994-1304101229.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= b9f02ee542f99ae0fd8f291dd570c53a
RANDOM SEED=1669465435

node127.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.60
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      26301472 kB
Buffers:       1402536 kB
Cached:        4707780 kB
SwapCached:          0 kB
Active:        1787984 kB
Inactive:      4385748 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      26301472 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:             108 kB
Writeback:           0 kB
AnonPages:       63876 kB
Mapped:          14488 kB
Slab:           410948 kB
PageTables:       4500 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   186496 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71820 MiB
End job on node127 at 2011-04-29 20:40:47