Trace number 3249504

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.25 1212.11

General information on the benchmark

Namecrafted/mosoi/289/
289-unsat-17x30.cnf
MD5SUM3a506c7557d86d0d326e3b3cdee85b69
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2040
Number of clauses240210
Sum of the clauses size954720
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 23060
Number of clauses of size 30
Number of clauses of size 4237150
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.19/0.26	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3249504-1304063575.cnf 
0.19/0.26	c Using SATTIMEOUT of 4800
0.19/0.26	c Using SATRAM of 15500
0.19/0.26	c Size of subset node = 32
0.19/0.26	c Variables = 2040
0.19/0.26	c Clauses = 240210
0.19/0.26	c Literals = 954720
0.19/0.26	c Reduced size:
0.19/0.26	c Variables = 2040
0.19/0.26	c Clauses = 240210
0.19/0.26	c Literals = 954720
0.19/0.26	c a2
0.69/0.76	c a1
1.90/1.98	c Independent sub-problems = 1
1.90/1.98	c Sub-problem with 2040 variables and 240210 clauses
1.90/1.98	c using permutation method 4
1.90/1.98	c count = 1
1.90/1.98	c Ordering finished 1.940 seconds
1.90/1.98	c mapped 1.980 seconds
1.90/1.98	c permuted 1.980 seconds
1.90/1.98	c about to sort
2.00/2.02	c sorted
2.39/2.47	c clauses now 240210
2.39/2.47	c init finished 2.050 seconds
2.39/2.47	c tree built 2.470 seconds
2.39/2.47	c 
2.39/2.47	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
2.39/2.47	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
2.39/2.47	c ................................ | ..........                    |                                  2.5         0.0             
2.39/2.47	c ................................ | ..........                  1 |                                  2.5         0.4             
2.39/2.47	c ................................ | ..........                  2 |                                  2.5         0.8             
2.39/2.48	c ................................ | ..........                  3 |                                  2.5         1.2             
2.39/2.48	c ................................ | ..........                  4 |                                  2.5         1.6             
2.39/2.48	c ................................ | ..........                  5 |                                  2.5         2.0             
2.39/2.49	c ................................ | ..........               1000 |              1023                2.5       395.3          377
95.79/26.05	c ................................ | ..........            1558000 | 65295        344663             96.0     16229.2      5619785
191.73/50.28	c ................................ | ..........            2761000 | 128205       641981            192.1     14376.5      9920130
287.68/74.49	c ................................ | ..........            3839000 | 189008       966318            288.1     13327.5     13993963
384.01/98.71	c ................................ | ..........            4950000 | 245165       1271059           384.1     12888.6     17568960
479.95/122.92	c ................................ | *.........            6044000 | 305549       1582701           480.0     12590.6     19691633
575.95/147.11	c ................................ | *.........            7137000 | 363515       1840767           576.0     12390.2     23850549
671.92/171.32	c ................................ | *.........            8076000 | 417350       2106172           672.0     12017.9     26835723
767.87/195.54	c ................................ | *.........            9076000 | 465856       2355602           768.0     11816.9     30313403
863.76/219.77	c ................................ | *.........           10013000 | 513299       2588362           864.0     11588.7     32759644
959.71/243.98	c ................................ | **........           10978000 | 559892       2844150           960.0     11434.8     36025722
1055.71/268.17	c ................................ | **........           11823000 | 608437       3118753          1056.0     11195.8     38576218
1151.63/292.39	c ................................ | **........           12705000 | 649844       3357616          1152.0     11028.6     41049201
1248.02/316.60	c ................................ | **........           13558000 | 693207       3593994          1248.0     10863.5     42527842
1343.61/340.79	c ................................ | **........           14384000 | 734800       3828861          1344.0     10702.4     45717458
1440.00/365.00	c ................................ | ***.......           15229000 | 775225       4076218          1440.0     10575.6     48258647
1535.98/389.21	c ................................ | ***.......           16012000 | 815434       4277483          1536.0     10424.3     48915294
1632.04/413.40	c ................................ | ***.......           16840000 | 855348       4508082          1632.1     10318.2     51270570
1727.65/437.59	c ................................ | ***.......           17648000 | 894471       4752821          1728.0     10212.8     53465198
1823.98/461.81	c ................................ | ***.......           18577000 | 939620       5013405          1824.0     10184.5     53907627
1919.96/486.01	c ................................ | ****......           19628000 | 984108       5227608          1920.0     10222.8     55369948
2015.64/510.19	c ................................ | ****......           20596000 | 1024823      5435844          2016.0     10216.3     57404615
2111.73/534.39	c ................................ | ****......           21450000 | 1066519      5630533          2112.1     10155.8     60561996
2208.05/558.61	c ................................ | ****......           22188000 | 1107143      5851045          2208.1     10048.4     62185805
2303.98/582.81	c ................................ | ****......           23021000 | 1148416      6085585          2304.0      9991.6     62861864
2399.97/607.02	c ................................ | *****.....           23832000 | 1185704      6292330          2400.1      9929.7     63060915
2495.92/631.24	c ................................ | *****.....           24601000 | 1226069      6511063          2496.1      9855.8     63492372
2591.92/655.43	c ................................ | *****.....           25475000 | 1268147      6732978          2592.1      9828.1     63813543
2687.91/679.83	c ................................ | *****.....           26278000 | 1307462      6935833          2688.0      9776.0     64634165
2784.01/704.01	c ................................ | *****.....           27091000 | 1344833      7142953          2784.0      9730.8     65798617
2880.10/728.21	c ................................ | ******....           27863000 | 1382332      7354964          2880.1      9674.3     67298744
2975.70/752.38	c ................................ | ******....           28599000 | 1417224      7566781          2976.0      9609.8     68892441
3071.77/776.56	c ................................ | ******....           29309000 | 1451302      7782796          3072.0      9540.6     70365257
3167.79/800.77	c ................................ | ******....           30009000 | 1484739      7981781          3168.1      9472.3     71466588
3263.79/824.96	c ................................ | ******....           30736000 | 1518812      8193347          3264.0      9416.6     71466588
3359.75/849.17	c ................................ | *******...           31583000 | 1564579      8384798          3360.0      9399.6     72546010
3455.88/873.34	c ................................ | *******...           32220000 | 1612693      8557004          3456.1      9322.8     76437149
3551.64/897.49	c ................................ | *******...           32951000 | 1650849      8732587          3552.0      9276.7     80744853
3648.06/921.71	c ................................ | *******...           33604000 | 1689567      8931255          3648.1      9211.4     84384156
3744.09/945.91	c ................................ | *******...           34170000 | 1723425      9086229          3744.1      9126.3     85081171
3839.74/970.09	c ................................ | ********..           34676000 | 1755465      9245189          3840.1      9029.9     85936434
3935.89/994.25	c ................................ | ********..           35167000 | 1785022      9392878          3936.1      8934.4     87897425
4031.95/1018.43	c ................................ | ********..           35874000 | 1819496      9565551          4032.1      8897.2     89351369
4128.07/1042.61	c ................................ | ********..           36486000 | 1856691      9738508          4128.1      8838.4     91316054
4223.85/1066.76	c ................................ | ********..           37030000 | 1895456      9896281          4224.1      8766.3     93993891
4319.90/1090.93	c ................................ | *********.           37575000 | 1929208      10030848         4320.0      8697.8     94027047
4415.83/1115.15	c ................................ | *********.           38329000 | 1962193      10195551         4416.1      8679.5     94147183
4511.94/1139.33	c ................................ | *********.           39106000 | 1996896      10362527         4512.1      8666.9     94147183
4607.98/1163.51	c ................................ | *********.           39786000 | 2031773      10526704         4608.0      8634.1     94147183
4703.82/1187.75	c ................................ | *********.           40453000 | 2068005      10728429         4704.0      8599.6     94147183
4799.80/1211.97	c CPU limit (4800) reached
4799.80/1211.97	c Time = 4800.10 seconds
4799.80/1211.97	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3249504-1304063575/watcher-3249504-1304063575 -o /tmp/evaluation-result-3249504-1304063575/solver-3249504-1304063575 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3249504-1304063575.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.07 8.00 7.53 5/181 31660
/proc/meminfo: memFree=12508492/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=9408 CPUtime=0 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) R 31658 31660 31098 0 -1 4202496 428 0 0 0 0 0 0 0 25 0 1 0 517957214 9633792 370 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473628357 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/31660/statm: 2352 371 147 36 0 257 0

[startup+0.059323 s]
/proc/loadavg: 8.07 8.00 7.53 5/181 31660
/proc/meminfo: memFree=12508492/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=16420 CPUtime=0.05 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) R 31658 31660 31098 0 -1 4202496 2117 0 0 0 5 0 0 0 25 0 1 0 517957214 16814080 2059 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473628357 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/31660/statm: 4105 2060 148 36 0 2010 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 16420

[startup+0.100316 s]
/proc/loadavg: 8.07 8.00 7.53 5/181 31660
/proc/meminfo: memFree=12508492/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=21268 CPUtime=0.09 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) R 31658 31660 31098 0 -1 4202496 3337 0 0 0 9 0 0 0 25 0 1 0 517957214 21778432 3279 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473628357 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/31660/statm: 5317 3280 148 36 0 3222 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 21268

[startup+0.300283 s]
/proc/loadavg: 8.07 8.00 7.53 5/181 31660
/proc/meminfo: memFree=12508492/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=59992 CPUtime=0.29 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) R 31658 31660 31098 0 -1 4202496 12114 0 0 0 28 1 0 0 25 0 1 0 517957214 61431808 12056 33554432000 4194304 4338100 140733583932496 18446744073709551615 4244803 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/31660/statm: 14998 12056 151 36 0 12903 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 59992

[startup+0.700328 s]
/proc/loadavg: 8.07 8.00 7.53 5/181 31660
/proc/meminfo: memFree=12508492/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=104476 CPUtime=0.69 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) R 31658 31660 31098 0 -1 4202496 21232 0 0 0 67 2 0 0 25 0 1 0 517957214 106983424 20576 33554432000 4194304 4338100 140733583932496 18446744073709551615 4206993 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/31660/statm: 26119 20576 777 36 0 24024 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 104476

[startup+1.50118 s]
/proc/loadavg: 7.50 7.89 7.50 2/176 31662
/proc/meminfo: memFree=15069260/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=61196 CPUtime=1.49 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) R 31658 31660 31098 0 -1 4202496 25300 0 0 0 147 2 0 0 25 0 1 0 517957214 62664704 13313 33554432000 4194304 4338100 140733583932496 18446744073709551615 4245205 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/31660/statm: 15299 13313 185 36 0 13204 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 61196

[startup+3.10079 s]
/proc/loadavg: 7.50 7.89 7.50 2/176 31662
/proc/meminfo: memFree=15040120/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=342988 CPUtime=4.93 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) S 31658 31660 31098 0 -1 4202496 36468 0 0 0 483 10 0 0 25 0 5 0 517957214 351219712 23511 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 85747 23511 176 36 0 83652 0
Current children cumulated CPU time (s) 4.93
Current children cumulated vsize (KiB) 342988

[startup+6.30023 s]
/proc/loadavg: 7.50 7.89 7.50 9/186 31697
/proc/meminfo: memFree=14915408/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=374928 CPUtime=17.63 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) S 31658 31660 31098 0 -1 4202496 43619 0 0 0 1736 27 0 0 25 0 5 0 517957214 383926272 30662 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 93732 30662 176 36 0 91637 0
[pid=31660/tid=31663] ppid=31658 vsize=374928 CPUtime=3.78 cores=0,2,4,6
/proc/31660/task/31663/stat : 31663 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 2948 0 0 0 372 6 0 0 18 0 5 0 517957462 383926272 30662 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473362675 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=31660/tid=31664] ppid=31658 vsize=374928 CPUtime=3.75 cores=0,2,4,6
/proc/31660/task/31664/stat : 31664 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 2427 0 0 0 369 6 0 0 18 0 5 0 517957462 383926272 30662 33554432000 4194304 4338100 140733583932496 18446744073709551615 4265535 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=31660/tid=31665] ppid=31658 vsize=374928 CPUtime=3.78 cores=0,2,4,6
/proc/31660/task/31665/stat : 31665 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 2689 0 0 0 372 6 0 0 18 0 5 0 517957462 383926272 30662 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473362529 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=31660/tid=31666] ppid=31658 vsize=374928 CPUtime=3.8 cores=0,2,4,6
/proc/31660/task/31666/stat : 31666 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 1985 0 0 0 378 2 0 0 20 0 5 0 517957462 383926272 30662 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 17.63
Current children cumulated vsize (KiB) 374928

[startup+12.7011 s]
/proc/loadavg: 7.58 7.89 7.50 9/186 31697
/proc/meminfo: memFree=14847184/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=422904 CPUtime=43.04 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) S 31658 31660 31098 0 -1 4202496 56141 0 0 0 4229 75 0 0 25 0 5 0 517957214 433053696 43184 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 105726 43184 176 36 0 103631 0
[pid=31660/tid=31663] ppid=31658 vsize=422904 CPUtime=10.13 cores=0,2,4,6
/proc/31660/task/31663/stat : 31663 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 6136 0 0 0 994 19 0 0 20 0 5 0 517957462 433053696 43185 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473362857 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=31660/tid=31664] ppid=31658 vsize=422904 CPUtime=10.07 cores=0,2,4,6
/proc/31660/task/31664/stat : 31664 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 5719 0 0 0 987 20 0 0 25 0 5 0 517957462 433053696 43185 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473362466 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=31660/tid=31665] ppid=31658 vsize=422904 CPUtime=10.11 cores=0,2,4,6
/proc/31660/task/31665/stat : 31665 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 5996 0 0 0 992 19 0 0 19 0 5 0 517957462 433053696 43185 33554432000 4194304 4338100 140733583932496 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=31660/tid=31666] ppid=31658 vsize=422904 CPUtime=10.21 cores=0,2,4,6
/proc/31660/task/31666/stat : 31666 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 4721 0 0 0 1011 10 0 0 25 0 5 0 517957462 433053696 43185 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473363170 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 43.04
Current children cumulated vsize (KiB) 422904

[startup+25.5008 s]
/proc/loadavg: 7.72 7.91 7.51 9/186 31697
/proc/meminfo: memFree=14728044/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=512524 CPUtime=93.8 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) S 31658 31660 31098 0 -1 4202496 79902 0 0 0 9204 176 0 0 25 0 5 0 517957214 524824576 66945 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 128131 66945 176 36 0 126036 0
[pid=31660/tid=31663] ppid=31658 vsize=512524 CPUtime=22.81 cores=0,2,4,6
/proc/31660/task/31663/stat : 31663 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 11463 0 0 0 2229 52 0 0 19 0 5 0 517957462 524824576 66946 33554432000 4194304 4338100 140733583932496 18446744073709551615 4239479 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=31660/tid=31664] ppid=31658 vsize=512524 CPUtime=22.69 cores=0,2,4,6
/proc/31660/task/31664/stat : 31664 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 10791 0 0 0 2223 46 0 0 25 0 5 0 517957462 524824576 66946 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473362857 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=31660/tid=31665] ppid=31658 vsize=512524 CPUtime=22.78 cores=0,2,4,6
/proc/31660/task/31665/stat : 31665 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 14495 0 0 0 2232 46 0 0 19 0 5 0 517957462 524824576 66946 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=31660/tid=31666] ppid=31658 vsize=512524 CPUtime=23 cores=0,2,4,6
/proc/31660/task/31666/stat : 31666 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 9584 0 0 0 2275 25 0 0 25 0 5 0 517957462 524824576 66946 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 93.8
Current children cumulated vsize (KiB) 512524

[startup+51.1003 s]
/proc/loadavg: 7.81 7.92 7.53 10/186 31698
/proc/meminfo: memFree=14534680/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=660340 CPUtime=195.3 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) S 31658 31660 31098 0 -1 4202496 117567 0 0 0 19105 425 0 0 25 0 5 0 517957214 676188160 104610 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 165085 104610 176 36 0 162990 0
[pid=31660/tid=31663] ppid=31658 vsize=660340 CPUtime=48.13 cores=0,2,4,6
/proc/31660/task/31663/stat : 31663 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 19634 0 0 0 4691 122 0 0 22 0 5 0 517957462 676188160 104610 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473362900 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=31660/tid=31664] ppid=31658 vsize=660340 CPUtime=47.96 cores=0,2,4,6
/proc/31660/task/31664/stat : 31664 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 19969 0 0 0 4678 118 0 0 19 0 5 0 517957462 676188160 104610 33554432000 4194304 4338100 140733583932496 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=31660/tid=31665] ppid=31658 vsize=660340 CPUtime=48.1 cores=0,2,4,6
/proc/31660/task/31665/stat : 31665 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 26261 0 0 0 4694 116 0 0 18 0 5 0 517957462 676188160 104610 33554432000 4194304 4338100 140733583932496 18446744073709551615 4263846 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=31660/tid=31666] ppid=31658 vsize=660340 CPUtime=48.6 cores=0,2,4,6
/proc/31660/task/31666/stat : 31666 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 18133 0 0 0 4797 63 0 0 19 0 5 0 517957462 676188160 104610 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473363036 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 195.3
Current children cumulated vsize (KiB) 660340

[startup+102.306 s]
/proc/loadavg: 8.02 7.96 7.56 9/186 31699
/proc/meminfo: memFree=14206820/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=928240 CPUtime=398.29 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) S 31658 31660 31098 0 -1 4202496 184956 0 0 0 38861 968 0 0 25 0 5 0 517957214 950517760 171999 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 232060 171999 176 36 0 229965 0
[pid=31660/tid=31663] ppid=31658 vsize=928240 CPUtime=98.7 cores=0,2,4,6
/proc/31660/task/31663/stat : 31663 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 37216 0 0 0 9598 272 0 0 21 0 5 0 517957462 950517760 171999 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=31660/tid=31664] ppid=31658 vsize=928240 CPUtime=98.63 cores=0,2,4,6
/proc/31660/task/31664/stat : 31664 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 35147 0 0 0 9590 273 0 0 18 0 5 0 517957462 950517760 171999 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=31660/tid=31665] ppid=31658 vsize=928240 CPUtime=98.66 cores=0,2,4,6
/proc/31660/task/31665/stat : 31665 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 45004 0 0 0 9598 268 0 0 18 0 5 0 517957462 950517760 171999 33554432000 4194304 4338100 140733583932496 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=31660/tid=31666] ppid=31658 vsize=928240 CPUtime=99.79 cores=0,2,4,6
/proc/31660/task/31666/stat : 31666 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 34019 0 0 0 9829 150 0 0 20 0 5 0 517957462 950517760 171999 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 398.29
Current children cumulated vsize (KiB) 928240

[startup+162.301 s]
/proc/loadavg: 8.05 7.98 7.59 9/186 31701
/proc/meminfo: memFree=13900776/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=1178480 CPUtime=636.23 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) S 31658 31660 31098 0 -1 4202496 251474 0 0 0 62021 1602 0 0 25 0 5 0 517957214 1206763520 238517 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 294620 238517 176 36 0 292525 0
[pid=31660/tid=31663] ppid=31658 vsize=1178480 CPUtime=158.09 cores=0,2,4,6
/proc/31660/task/31663/stat : 31663 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 56174 0 0 0 15354 455 0 0 18 0 5 0 517957462 1206763520 238517 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=31660/tid=31664] ppid=31658 vsize=1178480 CPUtime=157.98 cores=0,2,4,6
/proc/31660/task/31664/stat : 31664 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 50858 0 0 0 15347 451 0 0 20 0 5 0 517957462 1206763520 238517 33554432000 4194304 4338100 140733583932496 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=31660/tid=31665] ppid=31658 vsize=1178480 CPUtime=157.89 cores=0,2,4,6
/proc/31660/task/31665/stat : 31665 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 66051 0 0 0 15344 445 0 0 25 0 5 0 517957462 1206763520 238517 33554432000 4194304 4338100 140733583932496 18446744073709551615 4256007 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=31660/tid=31666] ppid=31658 vsize=1178480 CPUtime=159.76 cores=0,2,4,6
/proc/31660/task/31666/stat : 31666 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 44821 0 0 0 15731 245 0 0 25 0 5 0 517957462 1206763520 238517 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 636.23
Current children cumulated vsize (KiB) 1178480

[startup+222.301 s]
/proc/loadavg: 8.12 8.01 7.62 9/186 31703
/proc/meminfo: memFree=13621380/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=1410612 CPUtime=874.05 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) S 31658 31660 31098 0 -1 4202496 311733 0 0 0 85142 2263 0 0 25 0 5 0 517957214 1444466688 298776 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 352653 298776 176 36 0 350558 0
[pid=31660/tid=31663] ppid=31658 vsize=1410612 CPUtime=217.44 cores=0,2,4,6
/proc/31660/task/31663/stat : 31663 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 71958 0 0 0 21099 645 0 0 25 0 5 0 517957462 1444466688 298776 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=31660/tid=31664] ppid=31658 vsize=1410612 CPUtime=217.14 cores=0,2,4,6
/proc/31660/task/31664/stat : 31664 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 67010 0 0 0 21076 638 0 0 19 0 5 0 517957462 1444466688 298776 33554432000 4194304 4338100 140733583932496 18446744073709551615 4254304 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=31660/tid=31665] ppid=31658 vsize=1410612 CPUtime=217.2 cores=0,2,4,6
/proc/31660/task/31665/stat : 31665 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 82294 0 0 0 21090 630 0 0 18 0 5 0 517957462 1444466688 298776 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=31660/tid=31666] ppid=31658 vsize=1410612 CPUtime=219.74 cores=0,2,4,6
/proc/31660/task/31666/stat : 31666 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 56901 0 0 0 21631 343 0 0 25 0 5 0 517957462 1444466688 298776 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 874.05
Current children cumulated vsize (KiB) 1410612

################
# More data... #
################

/proc/meminfo: memFree=12165028/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=2533252 CPUtime=2777.24 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) S 31658 31660 31098 0 -1 4202496 602934 0 0 0 270644 7080 0 0 25 0 5 0 517957214 2594050048 589977 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 633313 589977 176 36 0 631218 0
[pid=31660/tid=31663] ppid=31658 vsize=2533252 CPUtime=695.8 cores=0,2,4,6
/proc/31660/task/31663/stat : 31663 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 133948 0 0 0 68027 1553 0 0 18 0 5 0 517957462 2594050048 589977 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=31660/tid=31664] ppid=31658 vsize=2533252 CPUtime=691.68 cores=0,2,4,6
/proc/31660/task/31664/stat : 31664 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 139431 0 0 0 67186 1982 0 0 25 0 5 0 517957462 2594050048 589977 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=31660/tid=31665] ppid=31658 vsize=2533252 CPUtime=692.22 cores=0,2,4,6
/proc/31660/task/31665/stat : 31665 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 163086 0 0 0 67268 1954 0 0 18 0 5 0 517957462 2594050048 589977 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=31660/tid=31666] ppid=31658 vsize=2533252 CPUtime=695.04 cores=0,2,4,6
/proc/31660/task/31666/stat : 31666 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 132899 0 0 0 67918 1586 0 0 18 0 5 0 517957462 2594050048 589977 33554432000 4194304 4338100 140733583932496 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2777.24
Current children cumulated vsize (KiB) 2533252

[startup+762.306 s]
/proc/loadavg: 8.11 8.12 7.83 9/186 31729
/proc/meminfo: memFree=12010316/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=2648236 CPUtime=3015.41 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) S 31658 31660 31098 0 -1 4202496 634013 0 0 0 293841 7700 0 0 25 0 5 0 517957214 2711793664 621056 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 662059 621056 176 36 0 659964 0
[pid=31660/tid=31663] ppid=31658 vsize=2648236 CPUtime=755.77 cores=0,2,4,6
/proc/31660/task/31663/stat : 31663 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 144808 0 0 0 73921 1656 0 0 25 0 5 0 517957462 2711793664 621056 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473362479 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=31660/tid=31664] ppid=31658 vsize=2648236 CPUtime=751.16 cores=0,2,4,6
/proc/31660/task/31664/stat : 31664 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 140582 0 0 0 72957 2159 0 0 18 0 5 0 517957462 2711793664 621056 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473664026 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=31660/tid=31665] ppid=31658 vsize=2648236 CPUtime=751.63 cores=0,2,4,6
/proc/31660/task/31665/stat : 31665 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 173060 0 0 0 73036 2127 0 0 18 0 5 0 517957462 2711793664 621056 33554432000 4194304 4338100 140733583932496 18446744073709551615 4206915 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=31660/tid=31666] ppid=31658 vsize=2648236 CPUtime=754.33 cores=0,2,4,6
/proc/31660/task/31666/stat : 31666 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 141993 0 0 0 73681 1752 0 0 18 0 5 0 517957462 2711793664 621056 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473363182 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3015.41
Current children cumulated vsize (KiB) 2648236

[startup+822.301 s]
/proc/loadavg: 8.07 8.11 7.84 9/186 31731
/proc/meminfo: memFree=11907008/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=2719560 CPUtime=3253.46 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) S 31658 31660 31098 0 -1 4202496 652677 0 0 0 317045 8301 0 0 25 0 5 0 517957214 2784829440 639720 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 679890 639720 176 36 0 677795 0
[pid=31660/tid=31663] ppid=31658 vsize=2719560 CPUtime=815.32 cores=0,2,4,6
/proc/31660/task/31663/stat : 31663 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 149727 0 0 0 79739 1793 0 0 21 0 5 0 517957462 2784829440 639720 33554432000 4194304 4338100 140733583932496 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=31660/tid=31664] ppid=31658 vsize=2719560 CPUtime=810.63 cores=0,2,4,6
/proc/31660/task/31664/stat : 31664 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 143694 0 0 0 78734 2329 0 0 18 0 5 0 517957462 2784829440 639720 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=31660/tid=31665] ppid=31658 vsize=2719560 CPUtime=810.98 cores=0,2,4,6
/proc/31660/task/31665/stat : 31665 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 178328 0 0 0 78804 2294 0 0 18 0 5 0 517957462 2784829440 639720 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=31660/tid=31666] ppid=31658 vsize=2719560 CPUtime=814.01 cores=0,2,4,6
/proc/31660/task/31666/stat : 31666 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 147358 0 0 0 79522 1879 0 0 18 0 5 0 517957462 2784829440 639720 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3253.46
Current children cumulated vsize (KiB) 2719560

[startup+882.301 s]
/proc/loadavg: 8.15 8.12 7.86 9/186 31732
/proc/meminfo: memFree=11669156/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=2925256 CPUtime=3491.64 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) S 31658 31660 31098 0 -1 4202496 707081 0 0 0 340320 8844 0 0 25 0 5 0 517957214 2995462144 694124 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 731314 694124 176 36 0 729219 0
[pid=31660/tid=31663] ppid=31658 vsize=2925256 CPUtime=874.74 cores=0,2,4,6
/proc/31660/task/31663/stat : 31663 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 168166 0 0 0 85525 1949 0 0 20 0 5 0 517957462 2995462144 694124 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=31660/tid=31664] ppid=31658 vsize=2925256 CPUtime=870.1 cores=0,2,4,6
/proc/31660/task/31664/stat : 31664 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 152725 0 0 0 84529 2481 0 0 18 0 5 0 517957462 2995462144 694124 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=31660/tid=31665] ppid=31658 vsize=2925256 CPUtime=870.34 cores=0,2,4,6
/proc/31660/task/31665/stat : 31665 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 192377 0 0 0 84590 2444 0 0 19 0 5 0 517957462 2995462144 694124 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485781763 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=31660/tid=31666] ppid=31658 vsize=2925256 CPUtime=873.95 cores=0,2,4,6
/proc/31660/task/31666/stat : 31666 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 160243 0 0 0 85430 1965 0 0 19 0 5 0 517957462 2995462144 694124 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3491.64
Current children cumulated vsize (KiB) 2925256

[startup+942.301 s]
/proc/loadavg: 8.12 8.11 7.87 9/186 31734
/proc/meminfo: memFree=11414684/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=3146328 CPUtime=3729.77 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) S 31658 31660 31098 0 -1 4202496 763324 0 0 0 363566 9411 0 0 25 0 5 0 517957214 3221839872 750367 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 786582 750367 176 36 0 784487 0
[pid=31660/tid=31663] ppid=31658 vsize=3146328 CPUtime=934.16 cores=0,2,4,6
/proc/31660/task/31663/stat : 31663 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 181098 0 0 0 91306 2110 0 0 18 0 5 0 517957462 3221839872 750367 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473664043 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=31660/tid=31664] ppid=31658 vsize=3146328 CPUtime=929.55 cores=0,2,4,6
/proc/31660/task/31664/stat : 31664 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 164986 0 0 0 90316 2639 0 0 18 0 5 0 517957462 3221839872 750367 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473363182 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=31660/tid=31665] ppid=31658 vsize=3146328 CPUtime=930.08 cores=0,2,4,6
/proc/31660/task/31665/stat : 31665 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 209561 0 0 0 90441 2567 0 0 18 0 5 0 517957462 3221839872 750367 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473664043 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=31660/tid=31666] ppid=31658 vsize=3146328 CPUtime=933.47 cores=0,2,4,6
/proc/31660/task/31666/stat : 31666 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 174109 0 0 0 91257 2090 0 0 18 0 5 0 517957462 3221839872 750367 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473362681 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3729.77
Current children cumulated vsize (KiB) 3146328

[startup+1002.3 s]
/proc/loadavg: 8.17 8.13 7.89 9/186 31735
/proc/meminfo: memFree=11265484/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=3270036 CPUtime=3968.1 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) S 31658 31660 31098 0 -1 4202496 795778 0 0 0 386861 9949 0 0 25 0 5 0 517957214 3348516864 782821 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 817509 782821 176 36 0 815414 0
[pid=31660/tid=31663] ppid=31658 vsize=3270036 CPUtime=993.64 cores=0,2,4,6
/proc/31660/task/31663/stat : 31663 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 187740 0 0 0 97098 2266 0 0 25 0 5 0 517957462 3348516864 782821 33554432000 4194304 4338100 140733583932496 18446744073709551615 4229520 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=31660/tid=31664] ppid=31658 vsize=3270036 CPUtime=989.01 cores=0,2,4,6
/proc/31660/task/31664/stat : 31664 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 174488 0 0 0 96109 2792 0 0 19 0 5 0 517957462 3348516864 782821 33554432000 4194304 4338100 140733583932496 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=31660/tid=31665] ppid=31658 vsize=3270036 CPUtime=989.5 cores=0,2,4,6
/proc/31660/task/31665/stat : 31665 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 221576 0 0 0 96237 2713 0 0 18 0 5 0 517957462 3348516864 782821 33554432000 4194304 4338100 140733583932496 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=31660/tid=31666] ppid=31658 vsize=3270036 CPUtime=993.44 cores=0,2,4,6
/proc/31660/task/31666/stat : 31666 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 178404 0 0 0 97171 2173 0 0 23 0 5 0 517957462 3348516864 782821 33554432000 4194304 4338100 140733583932496 18446744073709551615 4255963 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3968.1
Current children cumulated vsize (KiB) 3270036

[startup+1062.3 s]
/proc/loadavg: 8.22 8.15 7.91 9/186 31737
/proc/meminfo: memFree=11070920/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=3431808 CPUtime=4206.34 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) S 31658 31660 31098 0 -1 4202496 837268 0 0 0 410171 10463 0 0 25 0 5 0 517957214 3514171392 824311 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 857952 824311 176 36 0 855857 0
[pid=31660/tid=31663] ppid=31658 vsize=3431808 CPUtime=1053.02 cores=0,2,4,6
/proc/31660/task/31663/stat : 31663 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 197539 0 0 0 102892 2410 0 0 18 0 5 0 517957462 3514171392 824311 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=31660/tid=31664] ppid=31658 vsize=3431808 CPUtime=1048.4 cores=0,2,4,6
/proc/31660/task/31664/stat : 31664 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 185776 0 0 0 101908 2932 0 0 18 0 5 0 517957462 3514171392 824311 33554432000 4194304 4338100 140733583932496 18446744073709551615 4283332 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=31660/tid=31665] ppid=31658 vsize=3431808 CPUtime=1049.05 cores=0,2,4,6
/proc/31660/task/31665/stat : 31665 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 233711 0 0 0 102050 2855 0 0 18 0 5 0 517957462 3514171392 824311 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=31660/tid=31666] ppid=31658 vsize=3431808 CPUtime=1053.35 cores=0,2,4,6
/proc/31660/task/31666/stat : 31666 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 186672 0 0 0 103076 2259 0 0 18 0 5 0 517957462 3514171392 824311 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4206.34
Current children cumulated vsize (KiB) 3431808

[startup+1122.3 s]
/proc/loadavg: 8.29 8.18 7.93 9/186 31739
/proc/meminfo: memFree=11026508/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=3460524 CPUtime=4444.41 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) S 31658 31660 31098 0 -1 4202496 843915 0 0 0 433480 10961 0 0 25 0 5 0 517957214 3543576576 830958 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 865131 830958 176 36 0 863036 0
[pid=31660/tid=31663] ppid=31658 vsize=3460524 CPUtime=1112.34 cores=0,2,4,6
/proc/31660/task/31663/stat : 31663 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 199219 0 0 0 108683 2551 0 0 18 0 5 0 517957462 3543576576 830958 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=31660/tid=31664] ppid=31658 vsize=3460524 CPUtime=1108.15 cores=0,2,4,6
/proc/31660/task/31664/stat : 31664 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 186259 0 0 0 107790 3025 0 0 25 0 5 0 517957462 3543576576 830958 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=31660/tid=31665] ppid=31658 vsize=3460524 CPUtime=1108.5 cores=0,2,4,6
/proc/31660/task/31665/stat : 31665 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 237693 0 0 0 107856 2994 0 0 18 0 5 0 517957462 3543576576 830958 33554432000 4194304 4338100 140733583932496 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=31660/tid=31666] ppid=31658 vsize=3460524 CPUtime=1112.91 cores=0,2,4,6
/proc/31660/task/31666/stat : 31666 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 187174 0 0 0 108906 2385 0 0 18 0 5 0 517957462 3543576576 830958 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4444.41
Current children cumulated vsize (KiB) 3460524

[startup+1182.3 s]
/proc/loadavg: 8.14 8.16 7.94 9/186 31740
/proc/meminfo: memFree=10967232/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=3462504 CPUtime=4682.5 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) S 31658 31660 31098 0 -1 4202496 846764 0 0 0 456732 11518 0 0 25 0 5 0 517957214 3545604096 833807 33554432000 4194304 4338100 140733583932496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 865626 833807 176 36 0 863531 0
[pid=31660/tid=31663] ppid=31658 vsize=3462504 CPUtime=1171.7 cores=0,2,4,6
/proc/31660/task/31663/stat : 31663 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 199739 0 0 0 114462 2708 0 0 19 0 5 0 517957462 3545604096 833807 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473362448 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=31660/tid=31664] ppid=31658 vsize=3462504 CPUtime=1168.09 cores=0,2,4,6
/proc/31660/task/31664/stat : 31664 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 186779 0 0 0 113695 3114 0 0 18 0 5 0 517957462 3545604096 833807 33554432000 4194304 4338100 140733583932496 18446744073709551615 4253933 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=31660/tid=31665] ppid=31658 vsize=3462504 CPUtime=1167.88 cores=0,2,4,6
/proc/31660/task/31665/stat : 31665 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 238213 0 0 0 113638 3150 0 0 18 0 5 0 517957462 3545604096 833807 33554432000 4194304 4338100 140733583932496 18446744073709551615 4239484 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=31660/tid=31666] ppid=31658 vsize=3462504 CPUtime=1172.33 cores=0,2,4,6
/proc/31660/task/31666/stat : 31666 (ttsth-5-4) R 31658 31660 31098 0 -1 4202560 188463 0 0 0 114692 2541 0 0 19 0 5 0 517957462 3545604096 833807 33554432000 4194304 4338100 140733583932496 18446744073709551615 274473362526 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4682.5
Current children cumulated vsize (KiB) 3462504



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1212 s]
/proc/loadavg: 7.88 8.11 7.93 2/173 31758
/proc/meminfo: memFree=12291432/32950928 swapFree=67111528/67111528
[pid=31660] ppid=31658 vsize=0 CPUtime=4800.13 cores=0,2,4,6
/proc/31660/stat : 31660 (ttsth-5-4) Z 31658 31660 31098 0 -1 4203548 847758 0 0 0 468241 11772 0 0 18 0 2 0 517957214 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31660/statm: 0 0 0 0 0 0 0
[pid=31660/tid=31664] ppid=31658 vsize=0 CPUtime=1197.73 cores=0,2,4,6
/proc/31660/task/31664/stat : 31664 (ttsth-5-4) R 31658 31660 31098 0 -1 4203588 187028 0 0 0 116615 3158 0 0 25 0 2 0 517957462 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.13
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 0
Real time (s): 1212.11
CPU time (s): 4800.25
CPU user time (s): 4682.42
CPU system time (s): 117.832
CPU usage (%): 396.024
Max. virtual memory (cumulated for all children) (KiB): 3463560

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4682.42
system time used= 117.832
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 847758
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1032735
involuntary context switches= 23833

runsolver used 1.74673 second user time and 4.02539 second system time

The end

Launcher Data 

Begin job on node125 at 2011-04-29 09:52:56
IDJOB=3249504
IDBENCH=83172
IDSOLVER=1654
FILE ID=node125/3249504-1304063575
RUNJOBID= node125-1304062369-31571
PBS_JOBID= 13170441
Free space on /tmp= 72760 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/mosoi/289/289-unsat-17x30.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3249504-1304063575/watcher-3249504-1304063575 -o /tmp/evaluation-result-3249504-1304063575/solver-3249504-1304063575 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3249504-1304063575.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 3a506c7557d86d0d326e3b3cdee85b69
RANDOM SEED=869259047

node125.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5266.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      12508656 kB
Buffers:       2219256 kB
Cached:       15023648 kB
SwapCached:          0 kB
Active:       11725500 kB
Inactive:      8181084 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      12508656 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            5864 kB
Writeback:           0 kB
AnonPages:     2662276 kB
Mapped:          15192 kB
Slab:           468532 kB
PageTables:       9852 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:  2894212 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72760 MiB
End job on node125 at 2011-04-29 10:13:10