Trace number 3259406

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.29 1202.9

General information on the benchmark

NameCRAFTED/rbsat/crafted/
forced/rbsat-v2100c215164gyes4.cnf
MD5SUM2caae162eff2e5f641cadfad345a5401
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2100
Number of clauses215164
Sum of the clauses size432288
Maximum clause length30
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2215094
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 570

Solver Data 

0.09/0.15	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259406-1304055274.cnf 
0.09/0.15	c Using SATTIMEOUT of 4800
0.09/0.15	c Using SATRAM of 15500
0.09/0.15	c Size of subset node = 32
0.09/0.15	c Variables = 2100
0.09/0.15	c Clauses = 215164
0.09/0.15	c Literals = 432288
0.09/0.15	c Reduced size:
0.09/0.15	c Variables = 2100
0.09/0.15	c Clauses = 215164
0.09/0.15	c Literals = 432288
0.09/0.15	c a2
0.19/0.27	c a1
0.49/0.55	c Independent sub-problems = 1
0.49/0.55	c Sub-problem with 2100 variables and 215164 clauses
0.49/0.55	c using permutation method 4
0.49/0.55	c count = 1
0.49/0.55	c Ordering finished 0.530 seconds
0.49/0.55	c mapped 0.550 seconds
0.49/0.55	c permuted 0.550 seconds
0.49/0.55	c about to sort
0.49/0.57	c sorted
1.10/1.10	c clauses now 215164
1.10/1.10	c init finished 0.610 seconds
1.10/1.10	c tree built 1.100 seconds
1.10/1.10	c 
1.10/1.10	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
1.10/1.10	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
1.10/1.10	c ................................ | ..........                    |                                  1.1         0.0             
1.10/1.10	c ................................ | ..........                  1 |                                  1.1         0.9             
1.10/1.10	c ................................ | ..........                  2 |                                  1.1         1.8             
1.10/1.10	c ................................ | ..........                  3 |                                  1.1         2.7             
1.10/1.10	c ................................ | ..........                  4 |                                  1.1         3.6             
1.10/1.10	c ................................ | ..........                  5 |                                  1.1         4.5             
1.10/1.11	c ................................ | ..........               1000 | 1            18                  1.1       909.1         1169
95.70/24.87	c ................................ | ..........            3382000 | 53425        293827             96.0     35221.8     17458520
191.93/48.91	c ................................ | ..........            6225000 | 96851        529347            192.0     32421.9     30080348
287.79/72.95	c ................................ | ..........            9131000 | 139589       730227            288.0     31703.8     38696297
383.64/96.99	c ................................ | ..........           11698000 | 183357       910539            384.0     30462.7     47423361
479.92/121.04	c ................................ | *.........           14047000 | 216596       1082929           480.1     29261.5     57438097
575.76/145.06	c ................................ | *.........           16173000 | 246194       1249947           576.0     28076.7     66585230
671.62/169.09	c ................................ | *.........           18368000 | 281368       1403998           672.0     27333.3     73799504
767.87/193.13	c ................................ | *.........           20364000 | 316508       1543492           768.0     26515.3     80764450
863.73/217.17	c ................................ | *.........           22264000 | 345603       1674270           864.0     25767.6     86532420
959.98/241.20	c ................................ | **........           23947000 | 373077       1801066           960.0     24944.8     90816334
1055.82/265.25	c ................................ | **........           25472000 | 402545       1920067          1056.0     24120.5     91842623
1151.70/289.28	c ................................ | **........           27563000 | 437862       2050923          1152.0     23926.2     96340773
1247.95/313.31	c ................................ | **........           29582000 | 470497       2170340          1248.0     23703.5    101355585
1343.80/337.37	c ................................ | **........           31290000 | 499230       2286325          1344.1     23279.5    102487890
1439.66/361.38	c ................................ | ***.......           33178000 | 533509       2399598          1440.0     23040.3    107546883
1535.92/385.42	c ................................ | ***.......           34901000 | 564721       2517754          1536.0     22721.6    111593731
1631.76/409.47	c ................................ | ***.......           36572000 | 590965       2632519          1632.0     22408.8    114677489
1728.02/433.58	c ................................ | ***.......           38310000 | 619910       2739015          1728.4     22165.5    119114520
1823.90/457.53	c ................................ | ***.......           40049000 | 654241       2845135          1824.0     21956.4    119554250
1919.72/481.57	c ................................ | ****......           41722000 | 681157       2949596          1920.0     21729.8    119877446
2015.96/505.61	c ................................ | ****......           43288000 | 709246       3050153          2016.0     21471.9    121104643
2111.81/529.65	c ................................ | ****......           44792000 | 738511       3143365          2112.0     21208.1    121950391
2207.65/553.69	c ................................ | ****......           46278000 | 767759       3235467          2208.0     20958.9    125808063
2303.90/577.73	c ................................ | ****......           47711000 | 797014       3332775          2304.0     20707.6    130212035
2399.79/601.77	c ................................ | *****.....           49067000 | 825266       3436506          2400.1     20444.0    136209387
2495.63/625.80	c ................................ | *****.....           50454000 | 848104       3548233          2496.0     20213.8    141296102
2591.88/649.84	c ................................ | *****.....           51952000 | 873516       3651789          2592.0     20042.9    145453614
2687.73/673.87	c ................................ | *****.....           53273000 | 899784       3751655          2688.0     19818.5    145453614
2783.98/697.90	c ................................ | *****.....           54561000 | 924546       3845519          2784.0     19598.0    145453614
2879.84/721.95	c ................................ | ******....           55898000 | 949562       3945220          2880.1     19408.7    147036879
2975.70/745.99	c ................................ | ******....           57310000 | 971246       4057774          2976.1     19257.0    151789570
3071.96/770.03	c ................................ | ******....           58569000 | 993390       4160751          3072.1     19065.1    153307886
3167.80/794.05	c ................................ | ******....           59734000 | 1014583      4245125          3168.0     18855.3    153519737
3263.65/818.09	c ................................ | ******....           60928000 | 1036277      4320611          3264.0     18666.6    154832917
3359.91/842.12	c ................................ | *******...           62269000 | 1059063      4402478          3360.0     18532.3    155590764
3455.77/866.16	c ................................ | *******...           63456000 | 1080260      4486626          3456.0     18360.9    158477139
3552.00/890.21	c ................................ | *******...           64575000 | 1100757      4576209          3552.1     18179.6    162611381
3647.86/914.23	c ................................ | *******...           65925000 | 1126004      4668120          3648.0     18071.4    166218710
3743.73/938.28	c ................................ | *******...           67121000 | 1148248      4749286          3744.1     17927.4    166457174
3839.99/962.31	c ................................ | ********..           68597000 | 1176158      4832858          3840.0     17863.6    169813007
3935.85/986.34	c ................................ | ********..           70065000 | 1203607      4924089          3936.0     17801.0    172755713
4031.71/1010.37	c ................................ | ********..           71509000 | 1227158      5011872          4032.0     17735.3    175576310
4127.99/1034.41	c ................................ | ********..           72710000 | 1247269      5097849          4128.0     17613.8    178714488
4223.84/1058.44	c ................................ | ********..           73843000 | 1267975      5172209          4224.0     17481.7    181417270
4319.73/1082.47	c ................................ | *********.           74957000 | 1289622      5247170          4320.0     17351.1    183777369
4415.99/1106.51	c ................................ | *********.           76026000 | 1311266      5321708          4416.0     17216.0    187606729
4511.85/1130.54	c ................................ | *********.           77065000 | 1332423      5393622          4512.0     17080.0    190288250
4607.72/1154.58	c ................................ | *********.           78291000 | 1352788      5465066          4608.1     16990.0    190804358
4703.98/1178.60	c ................................ | *********.           79469000 | 1371351      5536913          4704.0     16893.9    193445853
4799.84/1202.65	c CPU limit (4800) reached
4799.84/1202.65	c Time = 4800.03 seconds
4799.84/1202.65	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3259406-1304055274/watcher-3259406-1304055274 -o /tmp/evaluation-result-3259406-1304055274/solver-3259406-1304055274 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259406-1304055274.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.11 8.03 7.64 1/173 21466
/proc/meminfo: memFree=25143808/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=9760 CPUtime=0 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) R 21464 21466 18837 0 -1 4202496 489 0 0 0 0 0 0 0 23 0 1 0 517127095 9994240 431 33554432000 4194304 4338100 140736564517328 18446744073709551615 274664808192 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/21466/statm: 2440 432 147 36 0 345 0

[startup+0.098757 s]
/proc/loadavg: 8.11 8.03 7.64 1/173 21466
/proc/meminfo: memFree=25143808/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=28412 CPUtime=0.09 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) R 21464 21466 18837 0 -1 4202496 5088 0 0 0 9 0 0 0 24 0 1 0 517127095 29093888 5030 33554432000 4194304 4338100 140736564517328 18446744073709551615 274664469189 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/21466/statm: 7103 5032 148 36 0 5008 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 28412

[startup+0.100751 s]
/proc/loadavg: 8.11 8.03 7.64 1/173 21466
/proc/meminfo: memFree=25143808/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=28808 CPUtime=0.09 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) R 21464 21466 18837 0 -1 4202496 5185 0 0 0 9 0 0 0 24 0 1 0 517127095 29499392 5127 33554432000 4194304 4338100 140736564517328 18446744073709551615 274664469189 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/21466/statm: 7202 5128 148 36 0 5107 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 28808

[startup+0.300723 s]
/proc/loadavg: 8.11 8.03 7.64 1/173 21466
/proc/meminfo: memFree=25143808/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=56192 CPUtime=0.29 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) R 21464 21466 18837 0 -1 4202496 11715 0 0 0 28 1 0 0 25 0 1 0 517127095 57540608 11656 33554432000 4194304 4338100 140736564517328 18446744073709551615 4228247 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/21466/statm: 14048 11669 153 36 0 11953 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 56192

[startup+0.70065 s]
/proc/loadavg: 8.11 8.03 7.64 1/173 21466
/proc/meminfo: memFree=25143808/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=80516 CPUtime=0.69 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) R 21464 21466 18837 0 -1 4202496 19275 0 0 0 67 2 0 0 25 0 1 0 517127095 82448384 17953 33554432000 4194304 4338100 140736564517328 18446744073709551615 274664476929 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/21466/statm: 20129 17953 164 36 0 18034 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 80516

[startup+1.50048 s]
/proc/loadavg: 8.11 8.03 7.64 6/180 21498
/proc/meminfo: memFree=24988720/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=340592 CPUtime=2.65 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) S 21464 21466 18837 0 -1 4202496 41167 0 0 0 258 7 0 0 25 0 5 0 517127095 348766208 39424 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 85148 39425 176 36 0 83053 0
Current children cumulated CPU time (s) 2.65
Current children cumulated vsize (KiB) 340592

[startup+3.1012 s]
/proc/loadavg: 8.19 8.04 7.65 9/184 21502
/proc/meminfo: memFree=24881300/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=394692 CPUtime=9.05 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) S 21464 21466 18837 0 -1 4202496 53808 0 0 0 881 24 0 0 25 0 5 0 517127095 404164608 52065 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 98673 52065 176 36 0 96578 0
[pid=21466/tid=21499] ppid=21464 vsize=394692 CPUtime=1.99 cores=1,3,5,7
/proc/21466/task/21499/stat : 21499 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 5464 0 0 0 197 2 0 0 21 0 5 0 517127206 404164608 52065 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=21466/tid=21500] ppid=21464 vsize=394692 CPUtime=1.97 cores=1,3,5,7
/proc/21466/task/21500/stat : 21500 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 2951 0 0 0 190 7 0 0 21 0 5 0 517127206 404164608 52065 33554432000 4194304 4338100 140736564517328 18446744073709551615 274664203316 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=21466/tid=21501] ppid=21464 vsize=394692 CPUtime=1.98 cores=1,3,5,7
/proc/21466/task/21501/stat : 21501 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 5884 0 0 0 197 1 0 0 21 0 5 0 517127206 404164608 52065 33554432000 4194304 4338100 140736564517328 18446744073709551615 274664203745 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=21466/tid=21502] ppid=21464 vsize=394692 CPUtime=1.98 cores=1,3,5,7
/proc/21466/task/21502/stat : 21502 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 2865 0 0 0 190 8 0 0 21 0 5 0 517127206 404164608 52065 33554432000 4194304 4338100 140736564517328 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 9.05
Current children cumulated vsize (KiB) 394692

[startup+6.30065 s]
/proc/loadavg: 8.19 8.04 7.65 9/184 21502
/proc/meminfo: memFree=24677584/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=476492 CPUtime=21.82 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) S 21464 21466 18837 0 -1 4202496 75086 0 0 0 2129 53 0 0 25 0 5 0 517127095 487927808 73343 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 119123 73344 176 36 0 117028 0
[pid=21466/tid=21499] ppid=21464 vsize=476492 CPUtime=5.19 cores=1,3,5,7
/proc/21466/task/21499/stat : 21499 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 12520 0 0 0 515 4 0 0 25 0 5 0 517127206 487927808 73344 33554432000 4194304 4338100 140736564517328 18446744073709551615 274664203335 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=21466/tid=21500] ppid=21464 vsize=476492 CPUtime=5.15 cores=1,3,5,7
/proc/21466/task/21500/stat : 21500 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 6361 0 0 0 495 20 0 0 25 0 5 0 517127206 487927808 73344 33554432000 4194304 4338100 140736564517328 18446744073709551615 4286120 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=21466/tid=21501] ppid=21464 vsize=476492 CPUtime=5.17 cores=1,3,5,7
/proc/21466/task/21501/stat : 21501 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 13385 0 0 0 515 2 0 0 25 0 5 0 517127206 487927808 73345 33554432000 4194304 4338100 140736564517328 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=21466/tid=21502] ppid=21464 vsize=476492 CPUtime=5.18 cores=1,3,5,7
/proc/21466/task/21502/stat : 21502 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 6178 0 0 0 498 20 0 0 25 0 5 0 517127206 487927808 73345 33554432000 4194304 4338100 140736564517328 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 21.82
Current children cumulated vsize (KiB) 476492

[startup+12.7005 s]
/proc/loadavg: 8.16 8.04 7.65 9/184 21502
/proc/meminfo: memFree=24335388/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=639544 CPUtime=47.38 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) S 21464 21466 18837 0 -1 4202496 115294 0 0 0 4624 114 0 0 25 0 5 0 517127095 654893056 113551 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 159886 113551 176 36 0 157791 0
[pid=21466/tid=21499] ppid=21464 vsize=639544 CPUtime=11.59 cores=1,3,5,7
/proc/21466/task/21499/stat : 21499 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 24806 0 0 0 1151 8 0 0 25 0 5 0 517127206 654893056 113551 33554432000 4194304 4338100 140736564517328 18446744073709551615 274664203710 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=21466/tid=21500] ppid=21464 vsize=639544 CPUtime=11.51 cores=1,3,5,7
/proc/21466/task/21500/stat : 21500 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 12767 0 0 0 1104 47 0 0 25 0 5 0 517127206 654893056 113552 33554432000 4194304 4338100 140736564517328 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=21466/tid=21501] ppid=21464 vsize=639544 CPUtime=11.57 cores=1,3,5,7
/proc/21466/task/21501/stat : 21501 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 28579 0 0 0 1152 5 0 0 25 0 5 0 517127206 654893056 113552 33554432000 4194304 4338100 140736564517328 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=21466/tid=21502] ppid=21464 vsize=639544 CPUtime=11.58 cores=1,3,5,7
/proc/21466/task/21502/stat : 21502 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 12499 0 0 0 1111 47 0 0 25 0 5 0 517127206 654893056 113552 33554432000 4194304 4338100 140736564517328 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 47.38
Current children cumulated vsize (KiB) 639544

[startup+25.5003 s]
/proc/loadavg: 8.13 8.04 7.65 9/184 21503
/proc/meminfo: memFree=23788920/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=893504 CPUtime=98.5 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) S 21464 21466 18837 0 -1 4202496 179916 0 0 0 9614 236 0 0 25 0 5 0 517127095 914948096 178173 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 223376 178173 176 36 0 221281 0
[pid=21466/tid=21499] ppid=21464 vsize=893504 CPUtime=24.38 cores=1,3,5,7
/proc/21466/task/21499/stat : 21499 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 48052 0 0 0 2423 15 0 0 25 0 5 0 517127206 914948096 178174 33554432000 4194304 4338100 140736564517328 18446744073709551615 274664203280 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=21466/tid=21500] ppid=21464 vsize=893504 CPUtime=24.25 cores=1,3,5,7
/proc/21466/task/21500/stat : 21500 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 20385 0 0 0 2322 103 0 0 25 0 5 0 517127206 914948096 178174 33554432000 4194304 4338100 140736564517328 18446744073709551615 4285738 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=21466/tid=21501] ppid=21464 vsize=893504 CPUtime=24.35 cores=1,3,5,7
/proc/21466/task/21501/stat : 21501 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 53313 0 0 0 2426 9 0 0 25 0 5 0 517127206 914948096 178174 33554432000 4194304 4338100 140736564517328 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=21466/tid=21502] ppid=21464 vsize=893504 CPUtime=24.37 cores=1,3,5,7
/proc/21466/task/21502/stat : 21502 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 21523 0 0 0 2336 101 0 0 25 0 5 0 517127206 914948096 178174 33554432000 4194304 4338100 140736564517328 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 98.5
Current children cumulated vsize (KiB) 893504

[startup+51.1009 s]
/proc/loadavg: 8.15 8.05 7.67 9/184 21503
/proc/meminfo: memFree=23101824/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=1331716 CPUtime=200.72 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) S 21464 21466 18837 0 -1 4202496 288212 0 0 0 19607 465 0 0 25 0 5 0 517127095 1363677184 286469 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 332929 286469 176 36 0 330834 0
[pid=21466/tid=21499] ppid=21464 vsize=1331716 CPUtime=49.98 cores=1,3,5,7
/proc/21466/task/21499/stat : 21499 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 87232 0 0 0 4971 27 0 0 25 0 5 0 517127206 1363677184 286469 33554432000 4194304 4338100 140736564517328 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=21466/tid=21500] ppid=21464 vsize=1331716 CPUtime=49.71 cores=1,3,5,7
/proc/21466/task/21500/stat : 21500 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 34388 0 0 0 4764 207 0 0 25 0 5 0 517127206 1363677184 286469 33554432000 4194304 4338100 140736564517328 18446744073709551615 4256352 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=21466/tid=21501] ppid=21464 vsize=1331716 CPUtime=49.91 cores=1,3,5,7
/proc/21466/task/21501/stat : 21501 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 94725 0 0 0 4968 23 0 0 25 0 5 0 517127206 1363677184 286469 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=21466/tid=21502] ppid=21464 vsize=1331716 CPUtime=49.97 cores=1,3,5,7
/proc/21466/task/21502/stat : 21502 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 35224 0 0 0 4796 201 0 0 25 0 5 0 517127206 1363677184 286470 33554432000 4194304 4338100 140736564517328 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 200.72
Current children cumulated vsize (KiB) 1331716

[startup+102.307 s]
/proc/loadavg: 8.32 8.10 7.70 9/184 21505
/proc/meminfo: memFree=21922188/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=1917888 CPUtime=405.24 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) S 21464 21466 18837 0 -1 4202496 434312 0 0 0 39638 886 0 0 25 0 5 0 517127095 1963917312 430417 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 479472 430417 176 36 0 477377 0
[pid=21466/tid=21499] ppid=21464 vsize=1917888 CPUtime=101.19 cores=1,3,5,7
/proc/21466/task/21499/stat : 21499 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 143833 0 0 0 10072 47 0 0 25 0 5 0 517127206 1963917312 430417 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=21466/tid=21500] ppid=21464 vsize=1917888 CPUtime=100.63 cores=1,3,5,7
/proc/21466/task/21500/stat : 21500 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 53055 0 0 0 9662 401 0 0 18 0 5 0 517127206 1963917312 430417 33554432000 4194304 4338100 140736564517328 18446744073709551615 274664204007 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=21466/tid=21501] ppid=21464 vsize=1917888 CPUtime=101.1 cores=1,3,5,7
/proc/21466/task/21501/stat : 21501 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 146343 0 0 0 10078 32 0 0 25 0 5 0 517127206 1963917312 430417 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=21466/tid=21502] ppid=21464 vsize=1917888 CPUtime=101.18 cores=1,3,5,7
/proc/21466/task/21502/stat : 21502 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 54437 0 0 0 9719 399 0 0 25 0 5 0 517127206 1963917312 430417 33554432000 4194304 4338100 140736564517328 18446744073709551615 274664203387 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 405.24
Current children cumulated vsize (KiB) 1917888

[startup+162.301 s]
/proc/loadavg: 8.22 8.11 7.73 9/184 21506
/proc/meminfo: memFree=20783596/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=2620552 CPUtime=644.86 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) S 21464 21466 18837 0 -1 4202496 609260 0 0 0 63133 1353 0 0 25 0 5 0 517127095 2683445248 605365 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 655138 605365 176 36 0 653043 0
[pid=21466/tid=21499] ppid=21464 vsize=2620552 CPUtime=161.18 cores=1,3,5,7
/proc/21466/task/21499/stat : 21499 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 200621 0 0 0 16053 65 0 0 25 0 5 0 517127206 2683445248 605365 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254384 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=21466/tid=21500] ppid=21464 vsize=2620552 CPUtime=160.33 cores=1,3,5,7
/proc/21466/task/21500/stat : 21500 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 75935 0 0 0 15422 611 0 0 25 0 5 0 517127206 2683445248 605365 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254291 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=21466/tid=21501] ppid=21464 vsize=2620552 CPUtime=161.08 cores=1,3,5,7
/proc/21466/task/21501/stat : 21501 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 216959 0 0 0 16060 48 0 0 25 0 5 0 517127206 2683445248 605365 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=21466/tid=21502] ppid=21464 vsize=2620552 CPUtime=161.12 cores=1,3,5,7
/proc/21466/task/21502/stat : 21502 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 79101 0 0 0 15490 622 0 0 25 0 5 0 517127206 2683445248 605365 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254357 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 644.86
Current children cumulated vsize (KiB) 2620552

[startup+222.3 s]
/proc/loadavg: 8.14 8.11 7.75 9/184 21508
/proc/meminfo: memFree=19985640/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=3108520 CPUtime=884.49 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) S 21464 21466 18837 0 -1 4202496 730153 0 0 0 86694 1755 0 0 25 0 5 0 517127095 3183124480 726258 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 777130 726259 176 36 0 775035 0
[pid=21466/tid=21499] ppid=21464 vsize=3108520 CPUtime=221.17 cores=1,3,5,7
/proc/21466/task/21499/stat : 21499 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 246342 0 0 0 22033 84 0 0 25 0 5 0 517127206 3183124480 726259 33554432000 4194304 4338100 140736564517328 18446744073709551615 274664203373 0 0 4096 0 0 0 0 -1 3 0 0 0

################
# More data... #
################

/proc/21466/task/21502/stat : 21502 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 124189 0 0 0 67923 2082 0 0 25 0 5 0 517127206 5157412864 1205497 33554432000 4194304 4338100 140736564517328 18446744073709551615 4285491 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2801.55
Current children cumulated vsize (KiB) 5036536

[startup+762.307 s]
/proc/loadavg: 8.11 8.05 7.86 9/184 21522
/proc/meminfo: memFree=15827588/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=5249240 CPUtime=3041.2 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) S 21464 21466 18837 0 -1 4202496 1262636 0 0 0 299330 4790 0 0 25 0 5 0 517127095 5375221760 1258741 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 1312310 1258741 176 36 0 1310215 0
[pid=21466/tid=21499] ppid=21464 vsize=5249240 CPUtime=761.09 cores=1,3,5,7
/proc/21466/task/21499/stat : 21499 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 525417 0 0 0 75906 203 0 0 25 0 5 0 517127206 5375221760 1258741 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=21466/tid=21500] ppid=21464 vsize=5249240 CPUtime=758.14 cores=1,3,5,7
/proc/21466/task/21500/stat : 21500 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 119409 0 0 0 73585 2229 0 0 25 0 5 0 517127206 5375221760 1258741 33554432000 4194304 4338100 140736564517328 18446744073709551615 4253954 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=21466/tid=21501] ppid=21464 vsize=5249240 CPUtime=760.87 cores=1,3,5,7
/proc/21466/task/21501/stat : 21501 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 456977 0 0 0 75976 111 0 0 25 0 5 0 517127206 5375221760 1258741 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=21466/tid=21502] ppid=21464 vsize=5249240 CPUtime=759.94 cores=1,3,5,7
/proc/21466/task/21502/stat : 21502 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 124189 0 0 0 73756 2238 0 0 25 0 5 0 517127206 5375221760 1258741 33554432000 4194304 4338100 140736564517328 18446744073709551615 4253954 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3041.2
Current children cumulated vsize (KiB) 5249240

[startup+822.301 s]
/proc/loadavg: 8.11 8.06 7.87 9/184 21524
/proc/meminfo: memFree=15623212/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=5346208 CPUtime=3280.82 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) S 21464 21466 18837 0 -1 4202496 1287080 0 0 0 323029 5053 0 0 25 0 5 0 517127095 5474516992 1283185 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 1336552 1283185 176 36 0 1334457 0
[pid=21466/tid=21499] ppid=21464 vsize=5346208 CPUtime=821.08 cores=1,3,5,7
/proc/21466/task/21499/stat : 21499 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 534926 0 0 0 81897 211 0 0 25 0 5 0 517127206 5474516992 1283185 33554432000 4194304 4338100 140736564517328 18446744073709551615 4255947 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=21466/tid=21500] ppid=21464 vsize=5346208 CPUtime=818.13 cores=1,3,5,7
/proc/21466/task/21500/stat : 21500 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 119409 0 0 0 79455 2358 0 0 25 0 5 0 517127206 5474516992 1283185 33554432000 4194304 4338100 140736564517328 18446744073709551615 4257956 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=21466/tid=21501] ppid=21464 vsize=5346208 CPUtime=820.85 cores=1,3,5,7
/proc/21466/task/21501/stat : 21501 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 471912 0 0 0 81970 115 0 0 25 0 5 0 517127206 5474516992 1283185 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=21466/tid=21502] ppid=21464 vsize=5346208 CPUtime=819.61 cores=1,3,5,7
/proc/21466/task/21502/stat : 21502 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 124189 0 0 0 79599 2362 0 0 25 0 5 0 517127206 5474516992 1283185 33554432000 4194304 4338100 140736564517328 18446744073709551615 4206902 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3280.82
Current children cumulated vsize (KiB) 5346208

[startup+882.301 s]
/proc/loadavg: 8.09 8.06 7.88 9/184 21526
/proc/meminfo: memFree=15216956/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=5537016 CPUtime=3520.47 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) S 21464 21466 18837 0 -1 4202496 1334382 0 0 0 346741 5306 0 0 25 0 5 0 517127095 5669904384 1330487 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 1384254 1330487 176 36 0 1382159 0
[pid=21466/tid=21499] ppid=21464 vsize=5537016 CPUtime=881.05 cores=1,3,5,7
/proc/21466/task/21499/stat : 21499 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 555301 0 0 0 87884 221 0 0 25 0 5 0 517127206 5669904384 1330487 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=21466/tid=21500] ppid=21464 vsize=5537016 CPUtime=878.13 cores=1,3,5,7
/proc/21466/task/21500/stat : 21500 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 119409 0 0 0 85338 2475 0 0 25 0 5 0 517127206 5669904384 1330487 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=21466/tid=21501] ppid=21464 vsize=5537016 CPUtime=880.84 cores=1,3,5,7
/proc/21466/task/21501/stat : 21501 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 498839 0 0 0 87963 121 0 0 25 0 5 0 517127206 5669904384 1330487 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=21466/tid=21502] ppid=21464 vsize=5537016 CPUtime=879.31 cores=1,3,5,7
/proc/21466/task/21502/stat : 21502 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 124189 0 0 0 85450 2481 0 0 25 0 5 0 517127206 5669904384 1330487 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3520.47
Current children cumulated vsize (KiB) 5537016

[startup+942.301 s]
/proc/loadavg: 8.07 8.06 7.89 9/184 21527
/proc/meminfo: memFree=14850884/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=5712184 CPUtime=3760.09 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) S 21464 21466 18837 0 -1 4202496 1378598 0 0 0 370434 5575 0 0 25 0 5 0 517127095 5849276416 1374703 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 1428046 1374703 176 36 0 1425951 0
[pid=21466/tid=21499] ppid=21464 vsize=5712184 CPUtime=940.74 cores=1,3,5,7
/proc/21466/task/21499/stat : 21499 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 578977 0 0 0 93842 232 0 0 25 0 5 0 517127206 5849276416 1374703 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=21466/tid=21500] ppid=21464 vsize=5712184 CPUtime=938.12 cores=1,3,5,7
/proc/21466/task/21500/stat : 21500 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 119410 0 0 0 91210 2602 0 0 25 0 5 0 517127206 5849276416 1374703 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=21466/tid=21501] ppid=21464 vsize=5712184 CPUtime=940.82 cores=1,3,5,7
/proc/21466/task/21501/stat : 21501 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 519378 0 0 0 93957 125 0 0 25 0 5 0 517127206 5849276416 1374703 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254326 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=21466/tid=21502] ppid=21464 vsize=5712184 CPUtime=939.28 cores=1,3,5,7
/proc/21466/task/21502/stat : 21502 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 124189 0 0 0 91319 2609 0 0 25 0 5 0 517127206 5849276416 1374703 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3760.09
Current children cumulated vsize (KiB) 5712184

[startup+1002.3 s]
/proc/loadavg: 8.13 8.08 7.91 9/184 21529
/proc/meminfo: memFree=14474632/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=5978064 CPUtime=3999.75 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) S 21464 21466 18837 0 -1 4202496 1444696 0 0 0 394140 5835 0 0 25 0 5 0 517127095 6121537536 1440801 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 1494516 1440801 176 36 0 1492421 0
[pid=21466/tid=21499] ppid=21464 vsize=5978064 CPUtime=1000.43 cores=1,3,5,7
/proc/21466/task/21499/stat : 21499 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 618547 0 0 0 99794 249 0 0 25 0 5 0 517127206 6121537536 1440801 33554432000 4194304 4338100 140736564517328 18446744073709551615 4276415 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=21466/tid=21500] ppid=21464 vsize=5978064 CPUtime=998.11 cores=1,3,5,7
/proc/21466/task/21500/stat : 21500 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 119411 0 0 0 97094 2717 0 0 25 0 5 0 517127206 6121537536 1440801 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=21466/tid=21501] ppid=21464 vsize=5978064 CPUtime=1000.79 cores=1,3,5,7
/proc/21466/task/21501/stat : 21501 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 545905 0 0 0 99947 132 0 0 25 0 5 0 517127206 6121537536 1440801 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=21466/tid=21502] ppid=21464 vsize=5978064 CPUtime=999.28 cores=1,3,5,7
/proc/21466/task/21502/stat : 21502 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 124189 0 0 0 97198 2730 0 0 25 0 5 0 517127206 6121537536 1440801 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3999.75
Current children cumulated vsize (KiB) 5978064

[startup+1062.3 s]
/proc/loadavg: 8.15 8.09 7.92 9/184 21530
/proc/meminfo: memFree=14131252/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=6172000 CPUtime=4239.42 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) S 21464 21466 18837 0 -1 4202496 1493568 0 0 0 417872 6070 0 0 25 0 5 0 517127095 6320128000 1489673 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 1543000 1489673 176 36 0 1540905 0
[pid=21466/tid=21499] ppid=21464 vsize=6172000 CPUtime=1060.13 cores=1,3,5,7
/proc/21466/task/21499/stat : 21499 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 649949 0 0 0 105749 264 0 0 25 0 5 0 517127206 6320128000 1489673 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254285 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=21466/tid=21500] ppid=21464 vsize=6172000 CPUtime=1058.12 cores=1,3,5,7
/proc/21466/task/21500/stat : 21500 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 119411 0 0 0 102984 2828 0 0 25 0 5 0 517127206 6320128000 1489673 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=21466/tid=21501] ppid=21464 vsize=6172000 CPUtime=1060.77 cores=1,3,5,7
/proc/21466/task/21501/stat : 21501 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 563375 0 0 0 105940 137 0 0 25 0 5 0 517127206 6320128000 1489673 33554432000 4194304 4338100 140736564517328 18446744073709551615 274664204002 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=21466/tid=21502] ppid=21464 vsize=6172000 CPUtime=1059.27 cores=1,3,5,7
/proc/21466/task/21502/stat : 21502 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 124189 0 0 0 103092 2835 0 0 25 0 5 0 517127206 6320128000 1489673 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4239.42
Current children cumulated vsize (KiB) 6172000

[startup+1122.3 s]
/proc/loadavg: 8.14 8.10 7.93 9/184 21532
/proc/meminfo: memFree=13655948/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=6434752 CPUtime=4479.09 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) S 21464 21466 18837 0 -1 4202496 1560210 0 0 0 441608 6301 0 0 25 0 5 0 517127095 6589186048 1556315 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 1608688 1556315 176 36 0 1606593 0
[pid=21466/tid=21499] ppid=21464 vsize=6434752 CPUtime=1119.83 cores=1,3,5,7
/proc/21466/task/21499/stat : 21499 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 687804 0 0 0 111705 278 0 0 25 0 5 0 517127206 6589186048 1556315 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=21466/tid=21500] ppid=21464 vsize=6434752 CPUtime=1118.11 cores=1,3,5,7
/proc/21466/task/21500/stat : 21500 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 119411 0 0 0 108879 2932 0 0 25 0 5 0 517127206 6589186048 1556315 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=21466/tid=21501] ppid=21464 vsize=6434752 CPUtime=1120.75 cores=1,3,5,7
/proc/21466/task/21501/stat : 21501 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 589998 0 0 0 111932 143 0 0 25 0 5 0 517127206 6589186048 1556315 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=21466/tid=21502] ppid=21464 vsize=6434752 CPUtime=1119.26 cores=1,3,5,7
/proc/21466/task/21502/stat : 21502 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 126353 0 0 0 108985 2941 0 0 25 0 5 0 517127206 6589186048 1556315 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4479.09
Current children cumulated vsize (KiB) 6434752

[startup+1182.3 s]
/proc/loadavg: 8.14 8.11 7.94 9/184 21534
/proc/meminfo: memFree=13369484/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=6553616 CPUtime=4718.76 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) S 21464 21466 18837 0 -1 4202496 1589511 0 0 0 465380 6496 0 0 25 0 5 0 517127095 6710902784 1585616 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 1638404 1585616 176 36 0 1636309 0
[pid=21466/tid=21499] ppid=21464 vsize=6553616 CPUtime=1179.5 cores=1,3,5,7
/proc/21466/task/21499/stat : 21499 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 704285 0 0 0 117664 286 0 0 25 0 5 0 517127206 6710902784 1585616 33554432000 4194304 4338100 140736564517328 18446744073709551615 4240537 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=21466/tid=21500] ppid=21464 vsize=6553616 CPUtime=1178.12 cores=1,3,5,7
/proc/21466/task/21500/stat : 21500 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 119411 0 0 0 114788 3024 0 0 25 0 5 0 517127206 6710902784 1585616 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=21466/tid=21501] ppid=21464 vsize=6553616 CPUtime=1180.73 cores=1,3,5,7
/proc/21466/task/21501/stat : 21501 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 602817 0 0 0 117927 146 0 0 25 0 5 0 517127206 6710902784 1585616 33554432000 4194304 4338100 140736564517328 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=21466/tid=21502] ppid=21464 vsize=6553616 CPUtime=1179.27 cores=1,3,5,7
/proc/21466/task/21502/stat : 21502 (ttsth-5-4) R 21464 21466 18837 0 -1 4202560 126354 0 0 0 114894 3033 0 0 25 0 5 0 517127206 6710902784 1585616 33554432000 4194304 4338100 140736564517328 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4718.76
Current children cumulated vsize (KiB) 6553616



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.7 s]
/proc/loadavg: 8.17 8.12 7.95 6/181 21534
/proc/meminfo: memFree=14317968/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=0 CPUtime=4800.08 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) Z 21464 21466 18837 0 -1 4203548 1618998 0 0 0 473427 6581 0 0 18 0 2 0 517127095 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 0 0 0 0 0 0 0
[pid=21466/tid=21501] ppid=21464 vsize=0 CPUtime=1201.15 cores=1,3,5,7
/proc/21466/task/21501/stat : 21501 (ttsth-5-4) R 21464 21466 18837 0 -1 4203588 613233 0 0 0 119961 154 0 0 25 0 2 0 517127206 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.08
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202.81 s]
/proc/loadavg: 8.17 8.12 7.95 6/181 21534
/proc/meminfo: memFree=14317968/32951132 swapFree=67111524/67111528
[pid=21466] ppid=21464 vsize=0 CPUtime=4800.19 cores=1,3,5,7
/proc/21466/stat : 21466 (ttsth-5-4) Z 21464 21466 18837 0 -1 4203548 1618998 0 0 0 473427 6592 0 0 18 0 2 0 517127095 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/21466/statm: 0 0 0 0 0 0 0
[pid=21466/tid=21501] ppid=21464 vsize=0 CPUtime=1201.25 cores=1,3,5,7
/proc/21466/task/21501/stat : 21501 (ttsth-5-4) R 21464 21466 18837 0 -1 4203588 613233 0 0 0 119961 164 0 0 25 0 2 0 517127206 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.19
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1202.9
CPU time (s): 4800.29
CPU user time (s): 4734.28
CPU system time (s): 66.008
CPU usage (%): 399.061
Max. virtual memory (cumulated for all children) (KiB): 6672480

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4734.28
system time used= 66.008
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1618998
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4345
involuntary context switches= 33201

runsolver used 1.9847 second user time and 4.41633 second system time

The end

Launcher Data 

Begin job on node131 at 2011-04-29 07:34:35
IDJOB=3259406
IDBENCH=71571
IDSOLVER=1654
FILE ID=node131/3259406-1304055274
RUNJOBID= node131-1304054069-21378
PBS_JOBID= 13170364
Free space on /tmp= 71384 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/forced/rbsat-v2100c215164gyes4.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3259406-1304055274/watcher-3259406-1304055274 -o /tmp/evaluation-result-3259406-1304055274/solver-3259406-1304055274 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259406-1304055274.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 2caae162eff2e5f641cadfad345a5401
RANDOM SEED=1885763758

node131.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      25144088 kB
Buffers:        956828 kB
Cached:        6288316 kB
SwapCached:          4 kB
Active:         282876 kB
Inactive:      7030204 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      25144088 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:            2736 kB
Writeback:           0 kB
AnonPages:       67684 kB
Mapped:          15296 kB
Slab:           430808 kB
PageTables:       4788 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS:   220220 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71384 MiB
End job on node131 at 2011-04-29 07:54:39