Trace number 3257646

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.37 1204.34

General information on the benchmark

NameCRAFTED/modcircuits/
mod3block_2vars_11gates_u2_autoenc.cnf
MD5SUM36c3169a5cad0483dd15befcefc4323e
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark689.141
Satisfiable
(Un)Satisfiability was proved
Number of variables526
Number of clauses146535
Sum of the clauses size858954
Maximum clause length14
Minimum clause length1
Number of clauses of size 164
Number of clauses of size 22476
Number of clauses of size 34
Number of clauses of size 41387
Number of clauses of size 57328
Number of clauses of size over 5135276

Solver Data 

0.19/0.23	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3257646-1304046751.cnf 
0.19/0.23	c Using SATTIMEOUT of 4800
0.19/0.23	c Using SATRAM of 15500
0.19/0.23	c Size of subset node = 32
0.19/0.23	c Variables = 526
0.19/0.23	c Clauses = 146535
0.19/0.23	c Literals = 858954
0.19/0.23	c Reduced size:
0.19/0.23	c Variables = 526
0.19/0.23	c Clauses = 103871
0.19/0.23	c Literals = 567402
0.19/0.23	c a2
0.59/0.67	c a1
1.59/1.62	c Independent sub-problems = 1
1.59/1.62	c Sub-problem with 452 variables and 103871 clauses
1.59/1.62	c using permutation method 4
1.59/1.62	c count = 1
1.59/1.62	c Ordering finished 1.590 seconds
1.59/1.62	c mapped 1.610 seconds
1.59/1.62	c permuted 1.610 seconds
1.59/1.62	c about to sort
1.59/1.64	c sorted
1.79/1.85	c clauses now 103871
1.79/1.85	c init finished 1.650 seconds
1.79/1.85	c tree built 1.850 seconds
1.79/1.85	c 
1.79/1.85	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
1.79/1.85	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
1.79/1.85	c ................................ | ..........                    |                                  1.9         0.0             
1.79/1.85	c ................................ | ..........                  1 |                                  1.9         0.5             
1.79/1.85	c ................................ | ..........                  2 |                                  1.9         1.1             
1.79/1.85	c ................................ | ..........                  3 |                                  1.9         1.6             
1.79/1.85	c ................................ | ..........                  4 |                                  1.9         2.2             
1.79/1.85	c ................................ | ..........                  5 |                                  1.9         2.7             
1.79/1.87	c ................................ | ..........               1000 | 66           707                 1.9       523.6        27429
95.86/25.43	c ................................ | ..........             500000 | 72140        352431             96.0      5208.3     20136656
192.13/49.55	c ................................ | ..........             872000 | 131163       637682            192.3      4534.3     35090277
287.95/73.57	c ................................ | ..........            1271000 | 186488       919008            288.2      4409.4     49820364
384.20/97.61	c ................................ | ..........            1596000 | 236392       1166614           384.3      4153.3     61700502
480.05/121.60	c ................................ | *.........            1930000 | 284925       1376471           480.1      4020.3     70586437
575.90/145.64	c ................................ | *.........            2254000 | 327352       1554330           576.1      3912.7     80145888
672.15/169.74	c ................................ | *.........            2555000 | 362292       1726192           672.3      3800.2     88371485
768.01/193.71	c ................................ | *.........            2841000 | 396118       1889896           768.1      3698.9     94077801
863.89/217.78	c ................................ | *.........            3104000 | 427360       2059612           864.2      3591.8     98193973
959.73/241.78	c ................................ | **........            3364000 | 457328       2230685           960.1      3503.9     99398196
1055.95/265.86	c ................................ | **........            3608000 | 487105       2395103          1056.2      3416.0    104849507
1151.79/289.88	c ................................ | **........            3846000 | 514126       2556667          1152.1      3338.2    110217014
1247.64/313.89	c ................................ | **........            4091000 | 542166       2713308          1248.0      3278.0    114862374
1344.27/338.02	c ................................ | **........            4305000 | 572420       2854347          1344.4      3202.2    119997055
1440.07/362.01	c ................................ | ***.......            4524000 | 609368       2980890          1440.2      3141.3    121666077
1535.93/386.08	c ................................ | ***.......            4739000 | 637138       3120616          1536.2      3084.8    123513343
1631.93/410.16	c ................................ | ***.......            4951000 | 663880       3245978          1632.2      3033.3    125237964
1727.73/434.17	c ................................ | ***.......            5159000 | 690629       3408521          1728.0      2985.5    131298146
1823.92/458.26	c ................................ | ***.......            5359000 | 715916       3603667          1824.2      2937.8    134483087
1920.10/482.39	c ................................ | ****......            5557000 | 740886       3793023          1920.5      2893.6    139787793
2015.92/506.37	c ................................ | ****......            5747000 | 764998       3971275          2016.2      2850.4    148519976
2112.04/530.49	c ................................ | ****......            5930000 | 788262       4132154          2112.4      2807.2    157398541
2208.26/554.53	c ................................ | ****......            6122000 | 812536       4281009          2208.4      2772.1    164093315
2304.06/578.52	c ................................ | ****......            6298000 | 836791       4425284          2304.2      2733.3    170123706
2399.82/602.56	c ................................ | *****.....            6531000 | 867298       4583831          2400.1      2721.1    175803592
2495.98/626.66	c ................................ | *****.....            6747000 | 899899       4737488          2496.2      2702.9    181252837
2592.14/650.76	c ................................ | *****.....            6952000 | 932587       4868923          2592.4      2681.7    185485112
2687.95/674.77	c ................................ | *****.....            7150000 | 965478       4986523          2688.2      2659.7    187789459
2783.75/698.79	c ................................ | *****.....            7339000 | 989602       5120264          2784.2      2636.0    192992088
2879.85/722.87	c ................................ | ******....            7541000 | 1015792      5265712          2880.2      2618.3    197334788
2976.13/746.93	c ................................ | ******....            7717000 | 1038002      5387835          2976.2      2592.9    203795475
3071.74/771.08	c ................................ | ******....            7893000 | 1057839      5501419          3072.0      2569.3    207850491
3168.21/795.24	c ................................ | ******....            8068000 | 1078993      5616973          3168.4      2546.4    212265310
3263.94/819.25	c ................................ | ******....            8234000 | 1099139      5729866          3264.2      2522.5    214614722
3360.14/843.31	c ................................ | *******...            8397000 | 1119338      5832821          3360.2      2499.0    214847802
3456.23/867.42	c ................................ | *******...            8562000 | 1139620      5938873          3456.3      2477.2    217404456
3552.08/891.43	c ................................ | *******...            8714000 | 1157909      6035209          3552.2      2453.1    217477936
3648.33/915.52	c ................................ | *******...            8872000 | 1177904      6135821          3648.4      2431.7    219704910
3743.72/939.49	c ................................ | *******...            9025000 | 1195852      6237457          3744.1      2410.5    221383037
3840.37/963.67	c ................................ | ********..            9192000 | 1215831      6334233          3840.6      2393.4    224313007
3936.14/987.62	c ................................ | ********..            9346000 | 1233500      6437360          3936.2      2374.4    227091015
4032.34/1011.73	c ................................ | ********..            9496000 | 1254161      6549862          4032.5      2354.9    227091015
4128.54/1035.82	c ................................ | ********..            9673000 | 1277403      6669614          4128.6      2342.9    228868451
4223.86/1059.75	c ................................ | ********..            9849000 | 1300298      6785270          4224.1      2331.6    231397167
4320.05/1083.83	c ................................ | *********.           10012000 | 1322224      6890409          4320.2      2317.5    233118203
4416.27/1107.97	c ................................ | *********.           10184000 | 1341599      6985135          4416.6      2305.9    235035535
4511.71/1131.89	c ................................ | *********.           10349000 | 1362078      7083752          4512.1      2293.6    238088003
4607.91/1155.99	c ................................ | *********.           10500000 | 1381018      7188623          4608.3      2278.5    240396689
4704.11/1180.06	c ................................ | *********.           10653000 | 1400737      7312127          4704.4      2264.5    243129123
4799.96/1204.02	c CPU limit (4800) reached
4799.96/1204.02	c Time = 4800.06 seconds
4799.96/1204.02	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3257646-1304046751/watcher-3257646-1304046751 -o /tmp/evaluation-result-3257646-1304046751/solver-3257646-1304046751 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3257646-1304046751.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.30 3.60 3.59 3/177 4061
/proc/meminfo: memFree=22197224/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=9408 CPUtime=0 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) R 4059 4061 1895 0 -1 4202496 402 0 0 0 0 0 0 0 21 0 1 0 497090471 9633792 345 33554432000 4194304 4338100 140733981552160 18446744073709551615 254261616384 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4061/statm: 2352 345 147 36 0 257 0

[startup+0.0214469 s]
/proc/loadavg: 3.30 3.60 3.59 3/177 4061
/proc/meminfo: memFree=22197224/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=10692 CPUtime=0.01 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) R 4059 4061 1895 0 -1 4202496 726 0 0 0 1 0 0 0 21 0 1 0 497090471 10948608 669 33554432000 4194304 4338100 140733981552160 18446744073709551615 254261277381 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4061/statm: 2673 670 147 36 0 578 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 10692

[startup+0.10046 s]
/proc/loadavg: 3.30 3.60 3.59 3/177 4061
/proc/meminfo: memFree=22197224/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=17608 CPUtime=0.09 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) R 4059 4061 1895 0 -1 4202496 2441 0 0 0 8 1 0 0 21 0 1 0 497090471 18030592 2384 33554432000 4194304 4338100 140733981552160 18446744073709551615 254261616384 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4061/statm: 4402 2385 148 36 0 2307 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 17608

[startup+0.300429 s]
/proc/loadavg: 3.30 3.60 3.59 3/177 4061
/proc/meminfo: memFree=22197224/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=76928 CPUtime=0.29 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) R 4059 4061 1895 0 -1 4202496 13952 0 0 0 28 1 0 0 22 0 1 0 497090471 78774272 13144 33554432000 4194304 4338100 140733981552160 18446744073709551615 254261012177 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4061/statm: 19232 13144 1155 36 0 17137 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 76928

[startup+0.700329 s]
/proc/loadavg: 3.30 3.60 3.59 3/177 4061
/proc/meminfo: memFree=22197224/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=28332 CPUtime=0.69 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) R 4059 4061 1895 0 -1 4202496 14414 0 0 0 67 2 0 0 24 0 1 0 497090471 29011968 5057 33554432000 4194304 4338100 140733981552160 18446744073709551615 4245205 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4061/statm: 7083 5059 176 36 0 4988 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 28332

[startup+1.50016 s]
/proc/loadavg: 3.30 3.60 3.59 7/182 4066
/proc/meminfo: memFree=22145740/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=39116 CPUtime=1.49 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) R 4059 4061 1895 0 -1 4202496 17151 0 0 0 147 2 0 0 25 0 1 0 497090471 40054784 7773 33554432000 4194304 4338100 140733981552160 18446744073709551615 4245192 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4061/statm: 9779 7773 155 36 0 7684 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 39116

[startup+3.10084 s]
/proc/loadavg: 3.30 3.60 3.59 9/186 4070
/proc/meminfo: memFree=22074284/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=306540 CPUtime=6.8 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) S 4059 4061 1895 0 -1 4202496 40042 0 0 0 672 8 0 0 25 0 5 0 497090471 313896960 30461 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273419237 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 76635 30462 176 36 0 74540 0
[pid=4061/tid=4067] ppid=4059 vsize=306540 CPUtime=1.23 cores=0,2,4,6
/proc/4061/task/4067/stat : 4067 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 5240 0 0 0 122 1 0 0 19 0 5 0 497090657 313896960 30462 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273429198 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=4061/tid=4068] ppid=4059 vsize=306540 CPUtime=1.23 cores=0,2,4,6
/proc/4061/task/4068/stat : 4068 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 3556 0 0 0 122 1 0 0 19 0 5 0 497090657 313896960 30462 33554432000 4194304 4338100 140733981552160 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=4061/tid=4069] ppid=4059 vsize=306540 CPUtime=1.23 cores=0,2,4,6
/proc/4061/task/4069/stat : 4069 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 3000 0 0 0 122 1 0 0 19 0 5 0 497090657 313896960 30462 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273442485 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=306540 CPUtime=1.22 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 5510 0 0 0 122 0 0 0 25 0 5 0 497090657 313896960 30463 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273430787 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 6.8
Current children cumulated vsize (KiB) 306540

[startup+6.30027 s]
/proc/loadavg: 3.68 3.68 3.62 9/186 4070
/proc/meminfo: memFree=21857484/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=435580 CPUtime=19.57 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) S 4059 4061 1895 0 -1 4202496 72582 0 0 0 1937 20 0 0 25 0 5 0 497090471 446033920 63001 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273419237 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 108895 63001 176 36 0 106800 0
[pid=4061/tid=4067] ppid=4059 vsize=435580 CPUtime=4.43 cores=0,2,4,6
/proc/4061/task/4067/stat : 4067 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 15613 0 0 0 441 2 0 0 25 0 5 0 497090657 446033920 63001 33554432000 4194304 4338100 140733981552160 18446744073709551615 254261011924 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=4061/tid=4068] ppid=4059 vsize=435580 CPUtime=4.43 cores=0,2,4,6
/proc/4061/task/4068/stat : 4068 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 9731 0 0 0 438 5 0 0 20 0 5 0 497090657 446033920 63001 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=4061/tid=4069] ppid=4059 vsize=435580 CPUtime=4.42 cores=0,2,4,6
/proc/4061/task/4069/stat : 4069 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 8812 0 0 0 437 5 0 0 25 0 5 0 497090657 446033920 63001 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273442485 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=435580 CPUtime=4.39 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 15689 0 0 0 437 2 0 0 25 0 5 0 497090657 446033920 63002 33554432000 4194304 4338100 140733981552160 18446744073709551615 4286307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 19.57
Current children cumulated vsize (KiB) 435580

[startup+12.7011 s]
/proc/loadavg: 4.02 3.75 3.64 9/186 4071
/proc/meminfo: memFree=21538488/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=620924 CPUtime=45.13 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) S 4059 4061 1895 0 -1 4202496 118568 0 0 0 4473 40 0 0 25 0 5 0 497090471 635826176 108987 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273419237 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 155231 108987 176 36 0 153136 0
[pid=4061/tid=4067] ppid=4059 vsize=620924 CPUtime=10.83 cores=0,2,4,6
/proc/4061/task/4067/stat : 4067 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 31811 0 0 0 1078 5 0 0 25 0 5 0 497090657 635826176 108987 33554432000 4194304 4338100 140733981552160 18446744073709551615 4241255 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=4061/tid=4068] ppid=4059 vsize=620924 CPUtime=10.83 cores=0,2,4,6
/proc/4061/task/4068/stat : 4068 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 17772 0 0 0 1069 14 0 0 25 0 5 0 497090657 635826176 108987 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273442485 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=4061/tid=4069] ppid=4059 vsize=620924 CPUtime=10.83 cores=0,2,4,6
/proc/4061/task/4069/stat : 4069 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 16220 0 0 0 1071 12 0 0 25 0 5 0 497090657 635826176 108987 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273442485 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=620924 CPUtime=10.77 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 30028 0 0 0 1072 5 0 0 25 0 5 0 497090657 635826176 108987 33554432000 4194304 4338100 140733981552160 18446744073709551615 254261012218 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 45.13
Current children cumulated vsize (KiB) 620924

[startup+25.5009 s]
/proc/loadavg: 4.90 3.95 3.71 9/186 4071
/proc/meminfo: memFree=21098040/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=883400 CPUtime=96.26 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) S 4059 4061 1895 0 -1 4202496 184430 0 0 0 9541 85 0 0 25 0 5 0 497090471 904601600 174849 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273419237 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 220850 174849 176 36 0 218755 0
[pid=4061/tid=4067] ppid=4059 vsize=883400 CPUtime=23.63 cores=0,2,4,6
/proc/4061/task/4067/stat : 4067 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 54816 0 0 0 2354 9 0 0 25 0 5 0 497090657 904601600 174849 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254285 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=4061/tid=4068] ppid=4059 vsize=883400 CPUtime=23.63 cores=0,2,4,6
/proc/4061/task/4068/stat : 4068 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 29313 0 0 0 2329 34 0 0 25 0 5 0 497090657 904601600 174849 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=4061/tid=4069] ppid=4059 vsize=883400 CPUtime=23.62 cores=0,2,4,6
/proc/4061/task/4069/stat : 4069 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 26016 0 0 0 2333 29 0 0 25 0 5 0 497090657 904601600 174849 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=883400 CPUtime=23.49 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 51548 0 0 0 2341 8 0 0 25 0 5 0 497090657 904601600 174849 33554432000 4194304 4338100 140733981552160 18446744073709551615 4287283 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 96.26
Current children cumulated vsize (KiB) 883400

[startup+51.1003 s]
/proc/loadavg: 5.96 4.28 3.82 9/186 4072
/proc/meminfo: memFree=20386968/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=1383736 CPUtime=198.5 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) S 4059 4061 1895 0 -1 4202496 310236 0 0 0 19695 155 0 0 25 0 5 0 497090471 1416945664 300655 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273419237 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 345934 300655 176 36 0 343839 0
[pid=4061/tid=4067] ppid=4059 vsize=1383736 CPUtime=49.22 cores=0,2,4,6
/proc/4061/task/4067/stat : 4067 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 91455 0 0 0 4906 16 0 0 25 0 5 0 497090657 1416945664 300655 33554432000 4194304 4338100 140733981552160 18446744073709551615 4255960 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=4061/tid=4068] ppid=4059 vsize=1383736 CPUtime=49.11 cores=0,2,4,6
/proc/4061/task/4068/stat : 4068 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 51365 0 0 0 4850 61 0 0 25 0 5 0 497090657 1416945664 300655 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254254 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=4061/tid=4069] ppid=4059 vsize=1383736 CPUtime=49.22 cores=0,2,4,6
/proc/4061/task/4069/stat : 4069 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 49244 0 0 0 4865 57 0 0 25 0 5 0 497090657 1416945664 300655 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=1383736 CPUtime=49.05 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 95435 0 0 0 4889 16 0 0 25 0 5 0 497090657 1416945664 300655 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 198.5
Current children cumulated vsize (KiB) 1383736

[startup+102.306 s]
/proc/loadavg: 7.19 4.87 4.04 9/186 4073
/proc/meminfo: memFree=19107156/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=2327476 CPUtime=402.98 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) S 4059 4061 1895 0 -1 4202496 529928 0 0 0 40032 266 0 0 25 0 5 0 497090471 2383335424 520347 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273419237 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 581869 520347 176 36 0 579774 0
[pid=4061/tid=4067] ppid=4059 vsize=2327476 CPUtime=100.41 cores=0,2,4,6
/proc/4061/task/4067/stat : 4067 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 157028 0 0 0 10010 31 0 0 25 0 5 0 497090657 2383335424 520347 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=4061/tid=4068] ppid=4059 vsize=2327476 CPUtime=100.03 cores=0,2,4,6
/proc/4061/task/4068/stat : 4068 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 92166 0 0 0 9899 104 0 0 25 0 5 0 497090657 2383335424 520347 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=4061/tid=4069] ppid=4059 vsize=2327476 CPUtime=100.41 cores=0,2,4,6
/proc/4061/task/4069/stat : 4069 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 93038 0 0 0 9943 98 0 0 25 0 5 0 497090657 2383335424 520347 33554432000 4194304 4338100 140733981552160 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=2327476 CPUtime=100.24 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 164959 0 0 0 9996 28 0 0 25 0 5 0 497090657 2383335424 520347 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 402.98
Current children cumulated vsize (KiB) 2327476

[startup+162.301 s]
/proc/loadavg: 7.80 5.46 4.30 9/186 4075
/proc/meminfo: memFree=17989660/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=3083488 CPUtime=642.61 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) S 4059 4061 1895 0 -1 4202496 720607 0 0 0 63902 359 0 0 25 0 5 0 497090471 3157491712 711026 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273419237 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 770872 711026 176 36 0 768777 0
[pid=4061/tid=4067] ppid=4059 vsize=3083488 CPUtime=160.4 cores=0,2,4,6
/proc/4061/task/4067/stat : 4067 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 239736 0 0 0 15991 49 0 0 25 0 5 0 497090657 3157491712 711026 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=4061/tid=4068] ppid=4059 vsize=3083488 CPUtime=159.69 cores=0,2,4,6
/proc/4061/task/4068/stat : 4068 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 127776 0 0 0 15830 139 0 0 25 0 5 0 497090657 3157491712 711026 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254239 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=4061/tid=4069] ppid=4059 vsize=3083488 CPUtime=160.4 cores=0,2,4,6
/proc/4061/task/4069/stat : 4069 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 122930 0 0 0 15910 130 0 0 25 0 5 0 497090657 3157491712 711026 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254357 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=3083488 CPUtime=160.23 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 207428 0 0 0 15987 36 0 0 25 0 5 0 497090657 3157491712 711026 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 642.61
Current children cumulated vsize (KiB) 3083488

[startup+222.301 s]
/proc/loadavg: 8.10 5.97 4.54 9/186 4076
/proc/meminfo: memFree=17240068/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=3470460 CPUtime=882.24 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) S 4059 4061 1895 0 -1 4202496 816824 0 0 0 87775 449 0 0 25 0 5 0 497090471 3553751040 807243 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273419237 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 867615 807243 176 36 0 865520 0
[pid=4061/tid=4067] ppid=4059 vsize=3470460 CPUtime=220.39 cores=0,2,4,6
/proc/4061/task/4067/stat : 4067 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 290731 0 0 0 21976 63 0 0 25 0 5 0 497090657 3553751040 807243 33554432000 4194304 4338100 140733981552160 18446744073709551615 4230663 0 0 4096 0 0 0 0 -1 0 0 0 0

################
# More data... #
################

/proc/4061/task/4067/stat : 4067 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 604488 0 0 0 75808 144 0 0 25 0 5 0 497090657 7103811584 1685906 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=4061/tid=4068] ppid=4059 vsize=6937316 CPUtime=757.97 cores=0,2,4,6
/proc/4061/task/4068/stat : 4068 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 299973 0 0 0 75401 396 0 0 25 0 5 0 497090657 7103811584 1685906 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=4061/tid=4069] ppid=4059 vsize=6937316 CPUtime=759.34 cores=0,2,4,6
/proc/4061/task/4069/stat : 4069 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 281242 0 0 0 75622 312 0 0 25 0 5 0 497090657 7103811584 1685906 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254352 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=6937316 CPUtime=758.69 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 487047 0 0 0 75776 93 0 0 25 0 5 0 497090657 7103811584 1685906 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3037.41
Current children cumulated vsize (KiB) 6937316

[startup+822.301 s]
/proc/loadavg: 8.33 7.94 6.28 9/186 4095
/proc/meminfo: memFree=11689836/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=7188876 CPUtime=3276.32 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) S 4059 4061 1895 0 -1 4202496 1761661 0 0 0 326639 993 0 0 25 0 5 0 497090471 7361409024 1752080 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273419237 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 1797219 1752080 176 36 0 1795124 0
[pid=4061/tid=4067] ppid=4059 vsize=7188876 CPUtime=819.31 cores=0,2,4,6
/proc/4061/task/4067/stat : 4067 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 632880 0 0 0 81778 153 0 0 25 0 5 0 497090657 7361409024 1752080 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254329 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=4061/tid=4068] ppid=4059 vsize=7188876 CPUtime=817.83 cores=0,2,4,6
/proc/4061/task/4068/stat : 4068 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 313041 0 0 0 81370 413 0 0 25 0 5 0 497090657 7361409024 1752080 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=4061/tid=4069] ppid=4059 vsize=7188876 CPUtime=819.28 cores=0,2,4,6
/proc/4061/task/4069/stat : 4069 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 293990 0 0 0 81603 325 0 0 25 0 5 0 497090657 7361409024 1752080 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=7188876 CPUtime=818.01 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 499013 0 0 0 81704 97 0 0 25 0 5 0 497090657 7361409024 1752080 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3276.32
Current children cumulated vsize (KiB) 7188876

[startup+882.301 s]
/proc/loadavg: 8.23 7.98 6.39 9/186 4097
/proc/meminfo: memFree=11569292/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=7277648 CPUtime=3515.73 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) S 4059 4061 1895 0 -1 4202496 1785894 0 0 0 350561 1012 0 0 25 0 5 0 497090471 7452311552 1776313 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273419237 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 1819412 1776313 176 36 0 1817317 0
[pid=4061/tid=4067] ppid=4059 vsize=7277648 CPUtime=879.27 cores=0,2,4,6
/proc/4061/task/4067/stat : 4067 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 638081 0 0 0 87772 155 0 0 25 0 5 0 497090657 7452311552 1776313 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254429 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=4061/tid=4068] ppid=4059 vsize=7277648 CPUtime=877.77 cores=0,2,4,6
/proc/4061/task/4068/stat : 4068 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 325094 0 0 0 87350 427 0 0 25 0 5 0 497090657 7452311552 1776313 33554432000 4194304 4338100 140733981552160 18446744073709551615 4287681 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=4061/tid=4069] ppid=4059 vsize=7277648 CPUtime=879.16 cores=0,2,4,6
/proc/4061/task/4069/stat : 4069 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 295228 0 0 0 87589 327 0 0 25 0 5 0 497090657 7452311552 1776313 33554432000 4194304 4338100 140733981552160 18446744073709551615 254261011910 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=7277648 CPUtime=877.65 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 504754 0 0 0 87667 98 0 0 25 0 5 0 497090657 7452311552 1776313 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3515.73
Current children cumulated vsize (KiB) 7277648

[startup+942.301 s]
/proc/loadavg: 8.16 8.00 6.50 9/186 4098
/proc/meminfo: memFree=11329340/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=7419728 CPUtime=3755.29 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) S 4059 4061 1895 0 -1 4202496 1822114 0 0 0 374499 1030 0 0 25 0 5 0 497090471 7597801472 1812533 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273419237 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 1854932 1812533 176 36 0 1852837 0
[pid=4061/tid=4067] ppid=4059 vsize=7419728 CPUtime=939.23 cores=0,2,4,6
/proc/4061/task/4067/stat : 4067 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 659546 0 0 0 93762 161 0 0 25 0 5 0 497090657 7597801472 1812533 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254433 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=4061/tid=4068] ppid=4059 vsize=7419728 CPUtime=937.73 cores=0,2,4,6
/proc/4061/task/4068/stat : 4068 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 325432 0 0 0 93340 433 0 0 25 0 5 0 497090657 7597801472 1812533 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=4061/tid=4069] ppid=4059 vsize=7419728 CPUtime=939.1 cores=0,2,4,6
/proc/4061/task/4069/stat : 4069 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 295577 0 0 0 93581 329 0 0 18 0 5 0 497090657 7597801472 1812533 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=7419728 CPUtime=937.35 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 518822 0 0 0 93632 103 0 0 25 0 5 0 497090657 7597801472 1812533 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3755.29
Current children cumulated vsize (KiB) 7419728

[startup+1002.3 s]
/proc/loadavg: 8.06 8.00 6.59 9/186 4100
/proc/meminfo: memFree=10999720/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=7583836 CPUtime=3994.8 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) S 4059 4061 1895 0 -1 4202496 1864818 0 0 0 398425 1055 0 0 25 0 5 0 497090471 7765848064 1855237 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273419237 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 1895959 1855237 176 36 0 1893864 0
[pid=4061/tid=4067] ppid=4059 vsize=7583836 CPUtime=999.2 cores=0,2,4,6
/proc/4061/task/4067/stat : 4067 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 686013 0 0 0 99753 167 0 0 25 0 5 0 497090657 7765848064 1855237 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=4061/tid=4068] ppid=4059 vsize=7583836 CPUtime=997.73 cores=0,2,4,6
/proc/4061/task/4068/stat : 4068 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 325804 0 0 0 99329 444 0 0 25 0 5 0 497090657 7765848064 1855237 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273442485 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=4061/tid=4069] ppid=4059 vsize=7583836 CPUtime=998.98 cores=0,2,4,6
/proc/4061/task/4069/stat : 4069 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 296053 0 0 0 99567 331 0 0 18 0 5 0 497090657 7765848064 1855237 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=7583836 CPUtime=996.99 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 534211 0 0 0 99592 107 0 0 25 0 5 0 497090657 7765848064 1855237 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3994.8
Current children cumulated vsize (KiB) 7583836

[startup+1062.3 s]
/proc/loadavg: 8.12 8.02 6.68 9/186 4102
/proc/meminfo: memFree=10758532/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=7726576 CPUtime=4234.23 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) S 4059 4061 1895 0 -1 4202496 1901432 0 0 0 422333 1090 0 0 25 0 5 0 497090471 7912013824 1891851 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273419237 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 1931644 1891851 176 36 0 1929549 0
[pid=4061/tid=4067] ppid=4059 vsize=7726576 CPUtime=1059.16 cores=0,2,4,6
/proc/4061/task/4067/stat : 4067 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 715133 0 0 0 105741 175 0 0 25 0 5 0 497090657 7912013824 1891851 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=4061/tid=4068] ppid=4059 vsize=7726576 CPUtime=1057.71 cores=0,2,4,6
/proc/4061/task/4068/stat : 4068 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 326175 0 0 0 105305 466 0 0 25 0 5 0 497090657 7912013824 1891851 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273442485 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=4061/tid=4069] ppid=4059 vsize=7726576 CPUtime=1058.8 cores=0,2,4,6
/proc/4061/task/4069/stat : 4069 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 296424 0 0 0 105546 334 0 0 18 0 5 0 497090657 7912013824 1891851 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273442485 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=7726576 CPUtime=1056.67 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 540963 0 0 0 105557 110 0 0 25 0 5 0 497090657 7912013824 1891851 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254439 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4234.23
Current children cumulated vsize (KiB) 7726576

[startup+1122.3 s]
/proc/loadavg: 8.22 8.06 6.78 9/186 4103
/proc/meminfo: memFree=10517700/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=7894076 CPUtime=4473.76 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) S 4059 4061 1895 0 -1 4202496 1944573 0 0 0 446256 1120 0 0 25 0 5 0 497090471 8083533824 1934992 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273419237 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 1973519 1934992 176 36 0 1971424 0
[pid=4061/tid=4067] ppid=4059 vsize=7894076 CPUtime=1119.17 cores=0,2,4,6
/proc/4061/task/4067/stat : 4067 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 753079 0 0 0 111732 185 0 0 25 0 5 0 497090657 8083533824 1934992 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=4061/tid=4068] ppid=4059 vsize=7894076 CPUtime=1117.71 cores=0,2,4,6
/proc/4061/task/4068/stat : 4068 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 329765 0 0 0 111289 482 0 0 25 0 5 0 497090657 8083533824 1934992 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254352 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=4061/tid=4069] ppid=4059 vsize=7894076 CPUtime=1118.67 cores=0,2,4,6
/proc/4061/task/4069/stat : 4069 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 296745 0 0 0 111529 338 0 0 18 0 5 0 497090657 8083533824 1934992 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254321 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=7894076 CPUtime=1116.33 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 542247 0 0 0 111522 111 0 0 25 0 5 0 497090657 8083533824 1934992 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4473.76
Current children cumulated vsize (KiB) 7894076

[startup+1182.3 s]
/proc/loadavg: 8.21 8.08 6.86 9/186 4105
/proc/meminfo: memFree=10207564/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=8102108 CPUtime=4713.3 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) S 4059 4061 1895 0 -1 4202496 1996962 0 0 0 470178 1152 0 0 25 0 5 0 497090471 8296558592 1987381 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273419237 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 2025527 1987381 176 36 0 2023432 0
[pid=4061/tid=4067] ppid=4059 vsize=8102108 CPUtime=1179.02 cores=0,2,4,6
/proc/4061/task/4067/stat : 4067 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 779149 0 0 0 117710 192 0 0 25 0 5 0 497090657 8296558592 1987381 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254362 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=4061/tid=4068] ppid=4059 vsize=8102108 CPUtime=1177.7 cores=0,2,4,6
/proc/4061/task/4068/stat : 4068 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 347615 0 0 0 117268 502 0 0 25 0 5 0 497090657 8296558592 1987381 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273442485 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=4061/tid=4069] ppid=4059 vsize=8102108 CPUtime=1178.55 cores=0,2,4,6
/proc/4061/task/4069/stat : 4069 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 302138 0 0 0 117513 342 0 0 18 0 5 0 497090657 8296558592 1987381 33554432000 4194304 4338100 140733981552160 18446744073709551615 254273442485 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=8102108 CPUtime=1176.15 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4202560 545323 0 0 0 117503 112 0 0 25 0 5 0 497090657 8296558592 1987381 33554432000 4194304 4338100 140733981552160 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4713.3
Current children cumulated vsize (KiB) 8102108



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1204.1 s]
/proc/loadavg: 7.82 8.01 6.87 2/177 4106
/proc/meminfo: memFree=15861144/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=0 CPUtime=4800.14 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) Z 4059 4061 1895 0 -1 4203548 2007349 0 0 0 478844 1170 0 0 18 0 2 0 497090471 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 0 0 0 0 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=0 CPUtime=1197.95 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4203588 548572 0 0 0 119674 121 0 0 25 0 2 0 497090657 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.14
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1204.21 s]
/proc/loadavg: 7.82 8.01 6.87 2/177 4106
/proc/meminfo: memFree=15861144/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=0 CPUtime=4800.24 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) Z 4059 4061 1895 0 -1 4203548 2007349 0 0 0 478844 1180 0 0 18 0 2 0 497090471 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 0 0 0 0 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=0 CPUtime=1198.05 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4203588 548572 0 0 0 119674 131 0 0 25 0 2 0 497090657 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 0

[startup+1204.3 s]
/proc/loadavg: 7.82 8.01 6.87 2/177 4106
/proc/meminfo: memFree=15861144/32951124 swapFree=67111528/67111528
[pid=4061] ppid=4059 vsize=0 CPUtime=4800.33 cores=0,2,4,6
/proc/4061/stat : 4061 (ttsth-5-4) Z 4059 4061 1895 0 -1 4203548 2007349 0 0 0 478844 1189 0 0 18 0 2 0 497090471 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/4061/statm: 0 0 0 0 0 0 0
[pid=4061/tid=4070] ppid=4059 vsize=0 CPUtime=1198.14 cores=0,2,4,6
/proc/4061/task/4070/stat : 4070 (ttsth-5-4) R 4059 4061 1895 0 -1 4203588 548572 0 0 0 119674 140 0 0 25 0 2 0 497090657 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.33
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1204.34
CPU time (s): 4800.37
CPU user time (s): 4788.44
CPU system time (s): 11.9342
CPU usage (%): 398.589
Max. virtual memory (cumulated for all children) (KiB): 8156240

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4788.44
system time used= 11.9342
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2007349
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 9291
involuntary context switches= 21801

runsolver used 2.06269 second user time and 4.52031 second system time

The end

Launcher Data 

Begin job on node140 at 2011-04-29 05:12:32
IDJOB=3257646
IDBENCH=71153
IDSOLVER=1654
FILE ID=node140/3257646-1304046751
RUNJOBID= node140-1304046751-4041
PBS_JOBID= 13170312
Free space on /tmp= 72656 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT09/CRAFTED/modcircuits/mod3block_2vars_11gates_u2_autoenc.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3257646-1304046751/watcher-3257646-1304046751 -o /tmp/evaluation-result-3257646-1304046751/solver-3257646-1304046751 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3257646-1304046751.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 36c3169a5cad0483dd15befcefc4323e
RANDOM SEED=995998144

node140.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.14
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      22198124 kB
Buffers:       2295924 kB
Cached:        7815548 kB
SwapCached:          0 kB
Active:        2651132 kB
Inactive:      7534224 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      22198124 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            5920 kB
Writeback:           0 kB
AnonPages:       72780 kB
Mapped:          14872 kB
Slab:           504192 kB
PageTables:       4876 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   215520 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72660 MiB
End job on node140 at 2011-04-29 05:32:38