Trace number 3264246

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.25 1202.49

General information on the benchmark

NameSAT04/handmade/kexu/frb-0.8/
frb59-26-1.used-as.sat04-891.cnf
MD5SUM5877850d9f604b6973676132cbba5c90
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1534
Number of clauses132295
Sum of the clauses size266006
Maximum clause length26
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2132236
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 559

Solver Data 

0.02/0.09	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3264246-1304099108.cnf 
0.02/0.09	c Using SATTIMEOUT of 4800
0.02/0.09	c Using SATRAM of 15500
0.02/0.09	c Size of subset node = 32
0.02/0.09	c Variables = 1534
0.02/0.09	c Clauses = 132295
0.02/0.09	c Literals = 266006
0.02/0.09	c Reduced size:
0.02/0.09	c Variables = 1534
0.02/0.09	c Clauses = 132295
0.02/0.09	c Literals = 266006
0.02/0.09	c a2
0.09/0.16	c a1
0.29/0.33	c Independent sub-problems = 1
0.29/0.33	c Sub-problem with 1534 variables and 132295 clauses
0.29/0.33	c using permutation method 4
0.29/0.33	c count = 1
0.29/0.33	c Ordering finished 0.300 seconds
0.29/0.33	c mapped 0.330 seconds
0.29/0.33	c permuted 0.330 seconds
0.29/0.33	c about to sort
0.29/0.34	c sorted
0.59/0.63	c clauses now 132295
0.59/0.63	c init finished 0.360 seconds
0.59/0.63	c tree built 0.630 seconds
0.59/0.63	c 
0.59/0.63	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.59/0.63	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.59/0.63	c ................................ | ..........                    |                                  0.6         0.0             
0.59/0.63	c ................................ | ..........                  1 |                                  0.6         1.6             
0.59/0.63	c ................................ | ..........                  2 |                                  0.6         3.2             
0.59/0.63	c ................................ | ..........                  3 |                                  0.6         4.8             
0.59/0.63	c ................................ | ..........                  4 |                                  0.6         6.3             
0.59/0.63	c ................................ | ..........                  5 |                                  0.6         7.9             
0.59/0.64	c ................................ | ..........               1000 | 1            106                 0.6      1562.5         2202
95.93/24.51	c ................................ | ..........            2885000 | 80985        281713             96.0     30052.1     20235419
191.79/48.56	c ................................ | ..........            4957000 | 134737       484558            192.0     25812.3     30609162
287.68/72.60	c ................................ | ..........            6667000 | 180747       643954            288.1     23145.3     38341521
383.93/96.62	c ................................ | ..........            8139000 | 218921       776561            384.0     21194.2     41697115
479.78/120.66	c ................................ | *.........            9511000 | 257403       892031            480.0     19813.8     50312610
575.66/144.69	c ................................ | *.........           10963000 | 296207       1010248           576.0     19031.7     59452506
671.92/168.72	c ................................ | *.........           12345000 | 336276       1126219           672.0     18370.3     67253282
767.79/192.75	c ................................ | *.........           13593000 | 373085       1244986           768.0     17698.8     73015390
864.05/216.80	c ................................ | *.........           14730000 | 406568       1347469           864.1     17047.0     73459112
959.94/240.83	c ................................ | **........           15822000 | 441581       1442075           960.1     16480.2     74223095
1055.79/264.86	c ................................ | **........           16823000 | 474477       1537141          1056.1     15930.0     75873395
1151.64/288.89	c ................................ | **........           17779000 | 504639       1616360          1152.0     15433.0     79133973
1247.89/312.93	c ................................ | **........           18849000 | 536861       1733828          1248.0     15102.8     82020892
1343.76/336.98	c ................................ | **........           19971000 | 572040       1837934          1344.1     14858.4     86610417
1440.03/361.01	c ................................ | ***.......           21049000 | 602156       1946519          1440.1     14616.4     91228992
1535.93/385.03	c ................................ | ***.......           22043000 | 635175       2036896          1536.1     14350.4     95014911
1631.76/409.06	c ................................ | ***.......           23050000 | 668581       2131504          1632.0     14123.8    102212495
1728.02/433.11	c ................................ | ***.......           24061000 | 703887       2214976          1728.1     13923.5    108290962
1823.88/457.15	c ................................ | ***.......           25023000 | 734780       2297009          1824.1     13717.9    114512216
1919.74/481.17	c ................................ | ****......           25959000 | 762541       2377163          1920.0     13520.2    120793542
2016.01/505.20	c ................................ | ****......           26902000 | 788158       2457247          2016.0     13344.0    126852386
2111.88/529.23	c ................................ | ****......           27706000 | 810280       2531164          2112.0     13118.4    128940568
2207.75/553.29	c ................................ | ****......           28531000 | 834485       2607048          2208.1     12921.1    133095051
2304.01/577.31	c ................................ | ****......           29386000 | 861270       2681653          2304.1     12754.0    134775140
2399.88/601.36	c ................................ | *****.....           30268000 | 885623       2753905          2400.1     12611.1    138049048
2495.74/625.37	c ................................ | *****.....           31123000 | 910080       2821755          2496.0     12469.0    141417948
2592.01/649.40	c ................................ | *****.....           32005000 | 940307       2895499          2592.0     12347.5    147117933
2687.88/673.43	c ................................ | *****.....           32832000 | 965469       2987616          2688.0     12214.1    152400473
2783.70/697.49	c ................................ | *****.....           33662000 | 987987       3076464          2784.1     12091.0    155580579
2879.97/721.53	c ................................ | ******....           34435000 | 1009644      3152802          2880.1     11956.2    156266898
2975.82/745.55	c ................................ | ******....           35142000 | 1029708      3225117          2976.1     11808.3    159287136
3072.09/769.60	c ................................ | ******....           35980000 | 1055562      3296946          3072.1     11711.9    163188203
3167.89/793.62	c ................................ | ******....           36757000 | 1080395      3366225          3168.0     11602.6    166327041
3263.76/817.67	c ................................ | ******....           37489000 | 1101877      3435989          3264.0     11485.5    167123958
3360.02/841.71	c ................................ | *******...           38156000 | 1120822      3505218          3360.1     11355.7    167504254
3455.90/865.75	c ................................ | *******...           38882000 | 1142100      3571103          3456.1     11250.4    168945646
3551.75/889.76	c ................................ | *******...           39514000 | 1159169      3633780          3552.0     11124.4    168945646
3648.02/913.81	c ................................ | *******...           40197000 | 1181958      3694421          3648.1     11018.7    169442629
3743.87/937.85	c ................................ | *******...           40911000 | 1202865      3754461          3744.1     10926.9    169533656
3839.68/961.89	c ................................ | ********..           41609000 | 1228052      3809755          3840.1     10835.5    169533656
3935.93/985.94	c ................................ | ********..           42332000 | 1253611      3867542          3936.1     10754.8    170233643
4031.78/1009.95	c ................................ | ********..           43031000 | 1275221      3922837          4032.0     10672.4    171929173
4127.67/1034.00	c ................................ | ********..           43738000 | 1296271      3983961          4128.1     10595.3    173653570
4223.93/1058.03	c ................................ | ********..           44423000 | 1316602      4043121          4224.1     10516.7    175404043
4319.79/1082.06	c ................................ | *********.           45635000 | 1350033      4118630          4320.1     10563.5    176853357
4415.66/1106.09	c ................................ | *********.           46783000 | 1380772      4190633          4416.0     10594.0    180388218
4511.93/1130.13	c ................................ | *********.           47952000 | 1410774      4265839          4512.1     10627.5    184359194
4607.76/1154.17	c ................................ | *********.           48959000 | 1438286      4337058          4608.1     10624.7    186460313
4704.00/1178.22	c ................................ | *********.           49998000 | 1462758      4416045          4704.1     10628.6    187803007
4799.82/1202.25	c CPU limit (4800) reached
4799.82/1202.25	c Time = 4800.01 seconds
4799.82/1202.25	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3264246-1304099108/watcher-3264246-1304099108 -o /tmp/evaluation-result-3264246-1304099108/solver-3264246-1304099108 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3264246-1304099108.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.88 7.83 7.84 5/185 10779
/proc/meminfo: memFree=24795280/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=9764 CPUtime=0 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) R 10777 10779 10639 0 -1 4202496 492 0 0 0 0 0 0 0 25 0 1 0 521509032 9998336 434 33554432000 4194304 4338100 140734336188016 18446744073709551615 246516495045 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/10779/statm: 2441 435 147 36 0 346 0

[startup+0.0215731 s]
/proc/loadavg: 7.88 7.83 7.84 5/185 10779
/proc/meminfo: memFree=24795280/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=12860 CPUtime=0.02 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) R 10777 10779 10639 0 -1 4202496 1274 0 0 0 2 0 0 0 25 0 1 0 521509032 13168640 1216 33554432000 4194304 4338100 140734336188016 18446744073709551615 246516495045 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/10779/statm: 3215 1217 148 36 0 1120 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 12860

[startup+0.100561 s]
/proc/loadavg: 7.88 7.83 7.84 5/185 10779
/proc/meminfo: memFree=24795280/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=30384 CPUtime=0.09 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) R 10777 10779 10639 0 -1 4202496 5645 0 0 0 9 0 0 0 25 0 1 0 521509032 31113216 5587 33554432000 4194304 4338100 140734336188016 18446744073709551615 246516495045 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/10779/statm: 7596 5590 151 36 0 5501 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 30384

[startup+0.300523 s]
/proc/loadavg: 7.88 7.83 7.84 5/185 10779
/proc/meminfo: memFree=24795280/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=38608 CPUtime=0.29 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) R 10777 10779 10639 0 -1 4202496 10343 0 0 0 29 0 0 0 25 0 1 0 521509032 39534592 7668 33554432000 4194304 4338100 140734336188016 18446744073709551615 4248092 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/10779/statm: 9652 7668 156 36 0 7557 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 38608

[startup+0.700415 s]
/proc/loadavg: 7.88 7.83 7.84 5/185 10779
/proc/meminfo: memFree=24795280/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=279504 CPUtime=0.87 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) S 10777 10779 10639 0 -1 4202496 26241 0 0 0 84 3 0 0 25 0 5 0 521509032 286212096 23048 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 69876 23050 176 36 0 67781 0
Current children cumulated CPU time (s) 0.87
Current children cumulated vsize (KiB) 279504

[startup+1.50027 s]
/proc/loadavg: 7.89 7.83 7.84 9/190 10784
/proc/meminfo: memFree=24665296/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=311708 CPUtime=4.07 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) S 10777 10779 10639 0 -1 4202496 34692 0 0 0 397 10 0 0 25 0 5 0 521509032 319188992 31499 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 77927 31499 176 36 0 75832 0
[pid=10779/tid=10781] ppid=10777 vsize=311708 CPUtime=0.85 cores=1,3,5,7
/proc/10779/task/10781/stat : 10781 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 3321 0 0 0 85 0 0 0 19 0 5 0 521509096 319188992 31499 33554432000 4194304 4338100 140734336188016 18446744073709551615 246516229214 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10779/tid=10782] ppid=10777 vsize=311708 CPUtime=0.85 cores=1,3,5,7
/proc/10779/task/10782/stat : 10782 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 2061 0 0 0 82 3 0 0 25 0 5 0 521509096 319188992 31499 33554432000 4194304 4338100 140734336188016 18446744073709551615 4256288 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=311708 CPUtime=0.85 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 2805 0 0 0 85 0 0 0 25 0 5 0 521509096 319188992 31499 33554432000 4194304 4338100 140734336188016 18446744073709551615 4283692 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10779/tid=10784] ppid=10777 vsize=311708 CPUtime=0.85 cores=1,3,5,7
/proc/10779/task/10784/stat : 10784 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 1911 0 0 0 83 2 0 0 25 0 5 0 521509096 319188992 31499 33554432000 4194304 4338100 140734336188016 18446744073709551615 4230401 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4.07
Current children cumulated vsize (KiB) 311708

[startup+3.10098 s]
/proc/loadavg: 7.89 7.83 7.84 9/190 10784
/proc/meminfo: memFree=24598984/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=371536 CPUtime=10.46 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) S 10777 10779 10639 0 -1 4202496 49163 0 0 0 1022 24 0 0 25 0 5 0 521509032 380452864 45970 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 92884 45970 176 36 0 90789 0
[pid=10779/tid=10781] ppid=10777 vsize=371536 CPUtime=2.45 cores=1,3,5,7
/proc/10779/task/10781/stat : 10781 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 8117 0 0 0 243 2 0 0 22 0 5 0 521509096 380452864 45970 33554432000 4194304 4338100 140734336188016 18446744073709551615 4256051 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10779/tid=10782] ppid=10777 vsize=371536 CPUtime=2.43 cores=1,3,5,7
/proc/10779/task/10782/stat : 10782 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 4477 0 0 0 235 8 0 0 25 0 5 0 521509096 380452864 45970 33554432000 4194304 4338100 140734336188016 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=371536 CPUtime=2.45 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 7770 0 0 0 244 1 0 0 25 0 5 0 521509096 380452864 45970 33554432000 4194304 4338100 140734336188016 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10779/tid=10784] ppid=10777 vsize=371536 CPUtime=2.45 cores=1,3,5,7
/proc/10779/task/10784/stat : 10784 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 4205 0 0 0 237 8 0 0 25 0 5 0 521509096 380452864 45970 33554432000 4194304 4338100 140734336188016 18446744073709551615 4240519 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 10.46
Current children cumulated vsize (KiB) 371536

[startup+6.30039 s]
/proc/loadavg: 7.89 7.83 7.84 9/190 10784
/proc/meminfo: memFree=24427828/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=472112 CPUtime=23.25 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) S 10777 10779 10639 0 -1 4202496 74607 0 0 0 2273 52 0 0 25 0 5 0 521509032 483442688 71414 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 118028 71415 176 36 0 115933 0
[pid=10779/tid=10781] ppid=10777 vsize=472112 CPUtime=5.65 cores=1,3,5,7
/proc/10779/task/10781/stat : 10781 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 17103 0 0 0 561 4 0 0 25 0 5 0 521509096 483442688 71415 33554432000 4194304 4338100 140734336188016 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10779/tid=10782] ppid=10777 vsize=472112 CPUtime=5.62 cores=1,3,5,7
/proc/10779/task/10782/stat : 10782 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 8665 0 0 0 542 20 0 0 25 0 5 0 521509096 483442688 71415 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=472112 CPUtime=5.65 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 15931 0 0 0 562 3 0 0 25 0 5 0 521509096 483442688 71416 33554432000 4194304 4338100 140734336188016 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10779/tid=10784] ppid=10777 vsize=472112 CPUtime=5.65 cores=1,3,5,7
/proc/10779/task/10784/stat : 10784 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 8317 0 0 0 545 20 0 0 25 0 5 0 521509096 483442688 71418 33554432000 4194304 4338100 140734336188016 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 23.25
Current children cumulated vsize (KiB) 472112

[startup+12.7012 s]
/proc/loadavg: 7.90 7.83 7.84 9/190 10784
/proc/meminfo: memFree=24140512/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=634412 CPUtime=48.8 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) S 10777 10779 10639 0 -1 4202496 114640 0 0 0 4774 106 0 0 25 0 5 0 521509032 649637888 111447 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 158603 111447 176 36 0 156508 0
[pid=10779/tid=10781] ppid=10777 vsize=634412 CPUtime=12.05 cores=1,3,5,7
/proc/10779/task/10781/stat : 10781 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 29914 0 0 0 1198 7 0 0 25 0 5 0 521509096 649637888 111447 33554432000 4194304 4338100 140734336188016 18446744073709551615 246516229167 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10779/tid=10782] ppid=10777 vsize=634412 CPUtime=11.98 cores=1,3,5,7
/proc/10779/task/10782/stat : 10782 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 15697 0 0 0 1153 45 0 0 25 0 5 0 521509096 649637888 111447 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=634412 CPUtime=12.05 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 29138 0 0 0 1200 5 0 0 25 0 5 0 521509096 649637888 111447 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254433 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10779/tid=10784] ppid=10777 vsize=634412 CPUtime=12.05 cores=1,3,5,7
/proc/10779/task/10784/stat : 10784 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 15297 0 0 0 1161 44 0 0 25 0 5 0 521509096 649637888 111447 33554432000 4194304 4338100 140734336188016 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 48.8
Current children cumulated vsize (KiB) 634412

[startup+25.5009 s]
/proc/loadavg: 8.07 7.87 7.85 9/190 10784
/proc/meminfo: memFree=23644392/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=940956 CPUtime=99.93 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) S 10777 10779 10639 0 -1 4202496 192580 0 0 0 9787 206 0 0 25 0 5 0 521509032 963538944 189387 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 235239 189387 176 36 0 233144 0
[pid=10779/tid=10781] ppid=10777 vsize=940956 CPUtime=24.85 cores=1,3,5,7
/proc/10779/task/10781/stat : 10781 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 60969 0 0 0 2471 14 0 0 25 0 5 0 521509096 963538944 189387 33554432000 4194304 4338100 140734336188016 18446744073709551615 4256753 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10779/tid=10782] ppid=10777 vsize=940956 CPUtime=24.71 cores=1,3,5,7
/proc/10779/task/10782/stat : 10782 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 27559 0 0 0 2384 87 0 0 25 0 5 0 521509096 963538944 189388 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254275 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=940956 CPUtime=24.85 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 52872 0 0 0 2475 10 0 0 25 0 5 0 521509096 963538944 189388 33554432000 4194304 4338100 140734336188016 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10779/tid=10784] ppid=10777 vsize=940956 CPUtime=24.84 cores=1,3,5,7
/proc/10779/task/10784/stat : 10784 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 26587 0 0 0 2394 90 0 0 25 0 5 0 521509096 963538944 189388 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 99.93
Current children cumulated vsize (KiB) 940956

[startup+51.1012 s]
/proc/loadavg: 8.23 7.93 7.87 9/190 10785
/proc/meminfo: memFree=22909388/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=1325584 CPUtime=202.19 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) S 10777 10779 10639 0 -1 4202496 284737 0 0 0 19836 383 0 0 25 0 5 0 521509032 1357398016 280936 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 331396 280936 176 36 0 329301 0
[pid=10779/tid=10781] ppid=10777 vsize=1325584 CPUtime=50.45 cores=1,3,5,7
/proc/10779/task/10781/stat : 10781 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 99411 0 0 0 5019 26 0 0 25 0 5 0 521509096 1357398016 280936 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10779/tid=10782] ppid=10777 vsize=1325584 CPUtime=50.24 cores=1,3,5,7
/proc/10779/task/10782/stat : 10782 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 37818 0 0 0 4859 165 0 0 25 0 5 0 521509096 1357398016 280936 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254347 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=1325584 CPUtime=50.45 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 85548 0 0 0 5029 16 0 0 25 0 5 0 521509096 1357398016 280937 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10779/tid=10784] ppid=10777 vsize=1325584 CPUtime=50.37 cores=1,3,5,7
/proc/10779/task/10784/stat : 10784 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 37366 0 0 0 4867 170 0 0 25 0 5 0 521509096 1357398016 280937 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 202.19
Current children cumulated vsize (KiB) 1325584

[startup+102.307 s]
/proc/loadavg: 8.44 8.03 7.91 9/190 10786
/proc/meminfo: memFree=22094304/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=1708916 CPUtime=406.72 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) S 10777 10779 10639 0 -1 4202496 378493 0 0 0 40015 657 0 0 25 0 5 0 521509032 1749929984 374692 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 427229 374692 176 36 0 425134 0
[pid=10779/tid=10781] ppid=10777 vsize=1708916 CPUtime=101.65 cores=1,3,5,7
/proc/10779/task/10781/stat : 10781 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 154859 0 0 0 10122 43 0 0 25 0 5 0 521509096 1749929984 374692 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10779/tid=10782] ppid=10777 vsize=1708916 CPUtime=101.44 cores=1,3,5,7
/proc/10779/task/10782/stat : 10782 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 38574 0 0 0 9852 292 0 0 25 0 5 0 521509096 1749929984 374692 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=1708916 CPUtime=101.65 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 123081 0 0 0 10140 25 0 0 25 0 5 0 521509096 1749929984 374692 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10779/tid=10784] ppid=10777 vsize=1708916 CPUtime=101.3 cores=1,3,5,7
/proc/10779/task/10784/stat : 10784 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 37385 0 0 0 9838 292 0 0 25 0 5 0 521509096 1749929984 374692 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254239 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 406.72
Current children cumulated vsize (KiB) 1708916

[startup+162.301 s]
/proc/loadavg: 8.46 8.09 7.93 9/190 10788
/proc/meminfo: memFree=20937468/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=2393948 CPUtime=646.36 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) S 10777 10779 10639 0 -1 4202496 549993 0 0 0 63668 968 0 0 25 0 5 0 521509032 2451402752 546192 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 598487 546192 176 36 0 596392 0
[pid=10779/tid=10781] ppid=10777 vsize=2393948 CPUtime=161.65 cores=1,3,5,7
/proc/10779/task/10781/stat : 10781 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 222499 0 0 0 16100 65 0 0 25 0 5 0 521509096 2451402752 546192 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10779/tid=10782] ppid=10777 vsize=2393948 CPUtime=161.43 cores=1,3,5,7
/proc/10779/task/10782/stat : 10782 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 62740 0 0 0 15712 431 0 0 25 0 5 0 521509096 2451402752 546192 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254321 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=2393948 CPUtime=161.64 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 180945 0 0 0 16127 37 0 0 25 0 5 0 521509096 2451402752 546192 33554432000 4194304 4338100 140734336188016 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10779/tid=10784] ppid=10777 vsize=2393948 CPUtime=160.96 cores=1,3,5,7
/proc/10779/task/10784/stat : 10784 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 59215 0 0 0 15666 430 0 0 25 0 5 0 521509096 2451402752 546192 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254362 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 646.36
Current children cumulated vsize (KiB) 2393948


################
# More data... #
################

/proc/10779/task/10784/stat : 10784 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 134525 0 0 0 68603 1407 0 0 25 0 5 0 521509096 5426786304 1269723 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2803.25
Current children cumulated vsize (KiB) 5299596

[startup+762.307 s]
/proc/loadavg: 8.20 8.12 8.01 9/190 10804
/proc/meminfo: memFree=15294264/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=5493532 CPUtime=3042.94 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) S 10777 10779 10639 0 -1 4202496 1322456 0 0 0 300982 3312 0 0 25 0 5 0 521509032 5625376768 1318655 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 1373383 1318655 176 36 0 1371288 0
[pid=10779/tid=10781] ppid=10777 vsize=5493532 CPUtime=759.91 cores=1,3,5,7
/proc/10779/task/10781/stat : 10781 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 575957 0 0 0 75802 189 0 0 25 0 5 0 521509096 5625376768 1318655 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254304 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10779/tid=10782] ppid=10777 vsize=5493532 CPUtime=761.4 cores=1,3,5,7
/proc/10779/task/10782/stat : 10782 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 145902 0 0 0 74627 1513 0 0 25 0 5 0 521509096 5625376768 1318655 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=5493532 CPUtime=760.85 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 441469 0 0 0 75978 107 0 0 25 0 5 0 521509096 5625376768 1318655 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10779/tid=10784] ppid=10777 vsize=5493532 CPUtime=760.11 cores=1,3,5,7
/proc/10779/task/10784/stat : 10784 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 134534 0 0 0 74513 1498 0 0 25 0 5 0 521509096 5625376768 1318655 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3042.94
Current children cumulated vsize (KiB) 5493532

[startup+822.301 s]
/proc/loadavg: 8.17 8.12 8.02 9/190 10806
/proc/meminfo: memFree=14847216/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=5656188 CPUtime=3282.53 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) S 10777 10779 10639 0 -1 4202496 1365334 0 0 0 324759 3494 0 0 25 0 5 0 521509032 5791936512 1361533 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 1414047 1361533 176 36 0 1411952 0
[pid=10779/tid=10781] ppid=10777 vsize=5656188 CPUtime=819.58 cores=1,3,5,7
/proc/10779/task/10781/stat : 10781 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 599188 0 0 0 81760 198 0 0 25 0 5 0 521509096 5791936512 1361533 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10779/tid=10782] ppid=10777 vsize=5656188 CPUtime=821.37 cores=1,3,5,7
/proc/10779/task/10782/stat : 10782 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 146027 0 0 0 80542 1595 0 0 25 0 5 0 521509096 5791936512 1361533 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=5656188 CPUtime=820.83 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 456712 0 0 0 81971 112 0 0 25 0 5 0 521509096 5791936512 1361533 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10779/tid=10784] ppid=10777 vsize=5656188 CPUtime=820.08 cores=1,3,5,7
/proc/10779/task/10784/stat : 10784 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 138813 0 0 0 80424 1584 0 0 25 0 5 0 521509096 5791936512 1361533 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3282.53
Current children cumulated vsize (KiB) 5656188

[startup+882.3 s]
/proc/loadavg: 8.11 8.11 8.02 10/190 10807
/proc/meminfo: memFree=14649912/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=5777060 CPUtime=3522.18 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) S 10777 10779 10639 0 -1 4202496 1386020 0 0 0 348571 3647 0 0 25 0 5 0 521509032 5915709440 1382219 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 1444265 1382219 176 36 0 1442170 0
[pid=10779/tid=10781] ppid=10777 vsize=5777060 CPUtime=879.23 cores=1,3,5,7
/proc/10779/task/10781/stat : 10781 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 608921 0 0 0 87720 203 0 0 25 0 5 0 521509096 5915709440 1382219 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10779/tid=10782] ppid=10777 vsize=5777060 CPUtime=881.38 cores=1,3,5,7
/proc/10779/task/10782/stat : 10782 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 149547 0 0 0 86468 1670 0 0 25 0 5 0 521509096 5915709440 1382219 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=5777060 CPUtime=880.82 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 463088 0 0 0 87968 114 0 0 25 0 5 0 521509096 5915709440 1382219 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10779/tid=10784] ppid=10777 vsize=5777060 CPUtime=880.09 cores=1,3,5,7
/proc/10779/task/10784/stat : 10784 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 139870 0 0 0 86353 1656 0 0 25 0 5 0 521509096 5915709440 1382219 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3522.18
Current children cumulated vsize (KiB) 5777060

[startup+942.3 s]
/proc/loadavg: 8.08 8.10 8.02 9/190 10809
/proc/meminfo: memFree=14380548/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=5836788 CPUtime=3761.82 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) S 10777 10779 10639 0 -1 4202496 1393018 0 0 0 372376 3806 0 0 25 0 5 0 521509032 5976870912 1389217 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 1459197 1389217 176 36 0 1457102 0
[pid=10779/tid=10781] ppid=10777 vsize=5836788 CPUtime=938.9 cores=1,3,5,7
/proc/10779/task/10781/stat : 10781 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 613530 0 0 0 93683 207 0 0 25 0 5 0 521509096 5976870912 1389217 33554432000 4194304 4338100 140734336188016 18446744073709551615 4235651 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10779/tid=10782] ppid=10777 vsize=5836788 CPUtime=941.37 cores=1,3,5,7
/proc/10779/task/10782/stat : 10782 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 150116 0 0 0 92391 1746 0 0 25 0 5 0 521509096 5976870912 1389217 33554432000 4194304 4338100 140734336188016 18446744073709551615 4235746 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=5836788 CPUtime=940.81 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 463094 0 0 0 93967 114 0 0 25 0 5 0 521509096 5976870912 1389217 33554432000 4194304 4338100 140734336188016 18446744073709551615 4235651 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10779/tid=10784] ppid=10777 vsize=5836788 CPUtime=940.08 cores=1,3,5,7
/proc/10779/task/10784/stat : 10784 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 141684 0 0 0 92273 1735 0 0 25 0 5 0 521509096 5976870912 1389217 33554432000 4194304 4338100 140734336188016 18446744073709551615 4235651 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3761.82
Current children cumulated vsize (KiB) 5836788

[startup+1002.3 s]
/proc/loadavg: 8.19 8.13 8.03 9/190 10819
/proc/meminfo: memFree=14053580/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=5889964 CPUtime=4001.43 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) S 10777 10779 10639 0 -1 4202496 1411529 0 0 0 396176 3967 0 0 25 0 5 0 521509032 6031323136 1407728 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 1472491 1407728 176 36 0 1470396 0
[pid=10779/tid=10781] ppid=10777 vsize=5889964 CPUtime=998.54 cores=1,3,5,7
/proc/10779/task/10781/stat : 10781 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 625578 0 0 0 99643 211 0 0 25 0 5 0 521509096 6031323136 1407728 33554432000 4194304 4338100 140734336188016 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10779/tid=10782] ppid=10777 vsize=5889964 CPUtime=1001.37 cores=1,3,5,7
/proc/10779/task/10782/stat : 10782 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 152730 0 0 0 98313 1824 0 0 25 0 5 0 521509096 6031323136 1407728 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=5889964 CPUtime=1000.78 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 464780 0 0 0 99964 114 0 0 25 0 5 0 521509096 6031323136 1407728 33554432000 4194304 4338100 140734336188016 18446744073709551615 4284167 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10779/tid=10784] ppid=10777 vsize=5889964 CPUtime=1000.08 cores=1,3,5,7
/proc/10779/task/10784/stat : 10784 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 143847 0 0 0 98195 1813 0 0 25 0 5 0 521509096 6031323136 1407728 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4001.43
Current children cumulated vsize (KiB) 5889964

[startup+1062.3 s]
/proc/loadavg: 8.17 8.13 8.03 9/190 10821
/proc/meminfo: memFree=13519492/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=6024468 CPUtime=4241.1 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) S 10777 10779 10639 0 -1 4202496 1445394 0 0 0 419964 4146 0 0 25 0 5 0 521509032 6169055232 1441593 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 1506117 1441593 176 36 0 1504022 0
[pid=10779/tid=10781] ppid=10777 vsize=6024468 CPUtime=1058.22 cores=1,3,5,7
/proc/10779/task/10781/stat : 10781 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 644340 0 0 0 105604 218 0 0 25 0 5 0 521509096 6169055232 1441593 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10779/tid=10782] ppid=10777 vsize=6024468 CPUtime=1061.36 cores=1,3,5,7
/proc/10779/task/10782/stat : 10782 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 152747 0 0 0 104228 1908 0 0 25 0 5 0 521509096 6169055232 1441593 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=6024468 CPUtime=1060.78 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 479856 0 0 0 105959 119 0 0 25 0 5 0 521509096 6169055232 1441593 33554432000 4194304 4338100 140734336188016 18446744073709551615 246516229243 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10779/tid=10784] ppid=10777 vsize=6024468 CPUtime=1060.07 cores=1,3,5,7
/proc/10779/task/10784/stat : 10784 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 143857 0 0 0 104110 1897 0 0 25 0 5 0 521509096 6169055232 1441593 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4241.1
Current children cumulated vsize (KiB) 6024468

[startup+1122.3 s]
/proc/loadavg: 8.16 8.14 8.04 9/190 10823
/proc/meminfo: memFree=13049740/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=6259068 CPUtime=4480.77 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) S 10777 10779 10639 0 -1 4202496 1503566 0 0 0 443751 4326 0 0 25 0 5 0 521509032 6409285632 1499765 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 1564767 1499765 176 36 0 1562672 0
[pid=10779/tid=10781] ppid=10777 vsize=6259068 CPUtime=1117.88 cores=1,3,5,7
/proc/10779/task/10781/stat : 10781 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 691112 0 0 0 111553 235 0 0 25 0 5 0 521509096 6409285632 1499765 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10779/tid=10782] ppid=10777 vsize=6259068 CPUtime=1121.36 cores=1,3,5,7
/proc/10779/task/10782/stat : 10782 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 152787 0 0 0 110149 1987 0 0 25 0 5 0 521509096 6409285632 1499765 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=6259068 CPUtime=1120.77 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 491206 0 0 0 111955 122 0 0 25 0 5 0 521509096 6409285632 1499765 33554432000 4194304 4338100 140734336188016 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10779/tid=10784] ppid=10777 vsize=6259068 CPUtime=1120.07 cores=1,3,5,7
/proc/10779/task/10784/stat : 10784 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 143867 0 0 0 110030 1977 0 0 25 0 5 0 521509096 6409285632 1499765 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4480.77
Current children cumulated vsize (KiB) 6259068

[startup+1182.3 s]
/proc/loadavg: 8.33 8.19 8.06 9/190 10827
/proc/meminfo: memFree=12762128/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=6409212 CPUtime=4720.37 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) S 10777 10779 10639 0 -1 4202496 1548926 0 0 0 467543 4494 0 0 25 0 5 0 521509032 6563033088 1545125 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 1602303 1545125 176 36 0 1600208 0
[pid=10779/tid=10781] ppid=10777 vsize=6409212 CPUtime=1177.53 cores=1,3,5,7
/proc/10779/task/10781/stat : 10781 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 720670 0 0 0 117507 246 0 0 25 0 5 0 521509096 6563033088 1545125 33554432000 4194304 4338100 140734336188016 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10779/tid=10782] ppid=10777 vsize=6409212 CPUtime=1181.35 cores=1,3,5,7
/proc/10779/task/10782/stat : 10782 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 156619 0 0 0 116069 2066 0 0 25 0 5 0 521509096 6563033088 1545125 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=6409212 CPUtime=1180.77 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 498959 0 0 0 117953 124 0 0 25 0 5 0 521509096 6563033088 1545125 33554432000 4194304 4338100 140734336188016 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10779/tid=10784] ppid=10777 vsize=6409212 CPUtime=1180.05 cores=1,3,5,7
/proc/10779/task/10784/stat : 10784 (ttsth-5-4) R 10777 10779 10639 0 -1 4202560 148084 0 0 0 115952 2053 0 0 25 0 5 0 521509096 6563033088 1545125 33554432000 4194304 4338100 140734336188016 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4720.37
Current children cumulated vsize (KiB) 6409212



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.31 s]
/proc/loadavg: 7.99 8.12 8.04 3/183 10856
/proc/meminfo: memFree=20066844/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=0 CPUtime=4800.07 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) Z 10777 10779 10639 0 -1 4203548 1557328 0 0 0 475458 4549 0 0 18 0 2 0 521509032 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 0 0 0 0 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=0 CPUtime=1200.79 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4203588 498960 0 0 0 119948 131 0 0 25 0 2 0 521509096 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.07
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202.41 s]
/proc/loadavg: 7.99 8.12 8.04 3/183 10856
/proc/meminfo: memFree=20066844/32951124 swapFree=67111352/67111528
[pid=10779] ppid=10777 vsize=0 CPUtime=4800.17 cores=1,3,5,7
/proc/10779/stat : 10779 (ttsth-5-4) Z 10777 10779 10639 0 -1 4203548 1557328 0 0 0 475458 4559 0 0 18 0 2 0 521509032 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/10779/statm: 0 0 0 0 0 0 0
[pid=10779/tid=10783] ppid=10777 vsize=0 CPUtime=1200.88 cores=1,3,5,7
/proc/10779/task/10783/stat : 10783 (ttsth-5-4) R 10777 10779 10639 0 -1 4203588 498960 0 0 0 119948 140 0 0 25 0 2 0 521509096 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.17
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1202.49
CPU time (s): 4800.25
CPU user time (s): 4754.59
CPU system time (s): 45.6631
CPU usage (%): 399.194
Max. virtual memory (cumulated for all children) (KiB): 6573052

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4754.59
system time used= 45.6631
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1557328
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3433
involuntary context switches= 33187

runsolver used 1.9687 second user time and 4.6063 second system time

The end

Launcher Data 

Begin job on node134 at 2011-04-29 19:45:08
IDJOB=3264246
IDBENCH=84568
IDSOLVER=1654
FILE ID=node134/3264246-1304099108
RUNJOBID= node134-1304097856-10658
PBS_JOBID= 13170616
Free space on /tmp= 71580 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT04/handmade/kexu/frb-0.8/frb59-26-1.used-as.sat04-891.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3264246-1304099108/watcher-3264246-1304099108 -o /tmp/evaluation-result-3264246-1304099108/solver-3264246-1304099108 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3264246-1304099108.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 5877850d9f604b6973676132cbba5c90
RANDOM SEED=490529117

node134.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 7186.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      24795436 kB
Buffers:       1491940 kB
Cached:        5858732 kB
SwapCached:         64 kB
Active:        1687768 kB
Inactive:      5946348 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      24795436 kB
SwapTotal:    67111528 kB
SwapFree:     67111352 kB
Dirty:            3528 kB
Writeback:           0 kB
AnonPages:      283304 kB
Mapped:          15376 kB
Slab:           459384 kB
PageTables:       4896 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   476976 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71584 MiB
End job on node134 at 2011-04-29 20:05:13