Trace number 3258658

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.23 1203.14

General information on the benchmark

NameCRAFTED/rbsat/random/
forced/rbsat-v2640c305320gyes5.cnf
MD5SUMf892a2bda6fe6aec6f2fcb4e1c541df8
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2640
Number of clauses305320
Sum of the clauses size613120
Maximum clause length33
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2305240
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 580

Solver Data 

0.19/0.21	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3258658-1304052151.cnf 
0.19/0.21	c Using SATTIMEOUT of 4800
0.19/0.21	c Using SATRAM of 15500
0.19/0.21	c Size of subset node = 32
0.19/0.21	c Variables = 2640
0.19/0.21	c Clauses = 305320
0.19/0.21	c Literals = 613120
0.19/0.21	c Reduced size:
0.19/0.21	c Variables = 2640
0.19/0.21	c Clauses = 305320
0.19/0.21	c Literals = 613120
0.19/0.21	c a2
0.29/0.38	c a1
0.79/0.81	c Independent sub-problems = 1
0.79/0.81	c Sub-problem with 2640 variables and 305320 clauses
0.79/0.81	c using permutation method 4
0.79/0.81	c count = 1
0.79/0.81	c Ordering finished 0.760 seconds
0.79/0.81	c mapped 0.800 seconds
0.79/0.81	c permuted 0.800 seconds
0.79/0.81	c about to sort
0.79/0.83	c sorted
1.59/1.60	c clauses now 305320
1.59/1.60	c init finished 0.890 seconds
1.59/1.60	c tree built 1.590 seconds
1.59/1.60	c 
1.59/1.60	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
1.59/1.60	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
1.59/1.60	c ................................ | ..........                    |                                  1.6         0.0             
1.59/1.60	c ................................ | ..........                  1 |                                  1.6         0.6             
1.59/1.60	c ................................ | ..........                  2 |                                  1.6         1.3             
1.59/1.60	c ................................ | ..........                  3 |                                  1.6         1.9             
1.59/1.60	c ................................ | ..........                  4 |                                  1.6         2.5             
1.59/1.60	c ................................ | ..........                  5 |                                  1.6         3.1             
1.59/1.60	c ................................ | ..........               1000 |              35                  1.6       625.0           67
95.86/25.23	c ................................ | ..........            3369000 | 75009        189995             96.0     35093.8     11106106
191.70/49.27	c ................................ | ..........            6229000 | 141404       340581            192.0     32442.7     19332766
287.98/73.30	c ................................ | ..........            8801000 | 200570       480092            288.0     30559.0     25318602
383.84/97.34	c ................................ | ..........           11050000 | 254982       604316            384.0     28774.5     27020412
479.71/121.38	c ................................ | *.........           13070000 | 304735       717497            480.1     27226.3     30625441
575.97/145.42	c ................................ | *.........           14935000 | 351262       818164            576.0     25926.6     33531317
671.87/169.45	c ................................ | *.........           16771000 | 397392       909269            672.0     24955.0     37546712
767.69/193.48	c ................................ | *.........           18719000 | 442375       1011080           768.0     24373.4     42034929
863.93/217.53	c ................................ | *.........           20461000 | 489216       1112099           864.1     23680.1     42544557
959.79/241.56	c ................................ | **........           21941000 | 528407       1196769           960.0     22854.5     44668621
1055.67/265.58	c ................................ | **........           23396000 | 567282       1279971          1056.0     22155.1     48567857
1151.93/289.62	c ................................ | **........           24868000 | 608777       1360594          1152.0     21586.2     52686707
1247.78/313.66	c ................................ | **........           26363000 | 644895       1444407          1248.0     21123.5     54709680
1343.68/337.69	c ................................ | **........           27834000 | 679881       1524168          1344.0     20709.7     58104679
1439.92/361.73	c ................................ | ***.......           29230000 | 716567       1597072          1440.1     20297.8     60432366
1535.80/385.76	c ................................ | ***.......           30570000 | 749428       1665889          1536.0     19902.0     61520280
1631.66/409.79	c ................................ | ***.......           31961000 | 785887       1732194          1632.1     19583.2     64907699
1727.93/433.83	c ................................ | ***.......           33351000 | 824685       1792234          1728.1     19299.6     67614746
1823.79/457.86	c ................................ | ***.......           34719000 | 858650       1856138          1824.1     19033.9     69006467
1919.68/481.89	c ................................ | ****......           36053000 | 890880       1920640          1920.0     18777.3     71546705
2015.91/505.93	c ................................ | ****......           37472000 | 918158       1996191          2016.0     18587.0     74917853
2111.77/529.96	c ................................ | ****......           38821000 | 949476       2057039          2112.0     18381.0     76487198
2208.03/554.00	c ................................ | ****......           40177000 | 979856       2120193          2208.0     18195.8     77907682
2303.91/578.02	c ................................ | ****......           41477000 | 1010094      2184327          2304.0     18002.2     79661692
2399.77/602.07	c ................................ | *****.....           42836000 | 1040345      2252129          2400.1     17848.0     79661692
2496.02/626.10	c ................................ | *****.....           44149000 | 1069823      2320825          2496.0     17687.7     80032552
2591.91/650.14	c ................................ | *****.....           45352000 | 1099109      2383500          2592.0     17496.6     81455070
2687.75/674.17	c ................................ | *****.....           46642000 | 1131686      2440082          2688.0     17351.7     84024057
2783.62/698.19	c ................................ | *****.....           47942000 | 1160043      2503256          2784.0     17220.5     85824260
2879.89/722.24	c ................................ | ******....           49251000 | 1190541      2566446          2880.1     17100.6     87135567
2975.76/746.27	c ................................ | ******....           50532000 | 1221129      2621782          2976.0     16979.6     89207548
3071.63/770.29	c ................................ | ******....           51820000 | 1251340      2674026          3072.0     16868.3     91108708
3167.91/794.32	c ................................ | ******....           53179000 | 1280296      2748074          3168.0     16786.2     94482723
3263.79/818.36	c ................................ | ******....           54438000 | 1309256      2804757          3264.0     16678.1     95487578
3359.63/842.39	c ................................ | *******...           55643000 | 1340425      2850616          3360.0     16560.3     98296731
3455.85/866.45	c ................................ | *******...           56810000 | 1370286      2895462          3456.1     16437.7     98442592
3551.70/890.47	c ................................ | *******...           57981000 | 1398342      2947533          3552.0     16323.5     98648208
3647.96/914.52	c ................................ | *******...           59167000 | 1426382      3003234          3648.0     16218.8    100832175
3743.83/938.55	c ................................ | *******...           60275000 | 1457322      3052862          3744.0     16099.0    103677708
3840.08/962.60	c ................................ | ********..           61429000 | 1486796      3098161          3840.1     15996.8    105603297
3935.95/986.64	c ................................ | ********..           62525000 | 1517689      3152776          3936.1     15884.9    108470485
4031.81/1010.65	c ................................ | ********..           63601000 | 1545202      3198751          4032.0     15774.0    110164432
4127.67/1034.69	c ................................ | ********..           64748000 | 1575076      3251142          4128.0     15684.9    110164432
4223.93/1058.72	c ................................ | ********..           65874000 | 1604137      3294494          4224.0     15595.0    110356216
4319.73/1082.78	c ................................ | *********.           66996000 | 1635470      3335618          4320.0     15508.2    111809108
4415.99/1106.80	c ................................ | *********.           68105000 | 1666047      3384833          4416.0     15422.3    112234716
4511.84/1130.85	c ................................ | *********.           69211000 | 1692086      3451622          4512.0     15339.2    114595791
4607.65/1154.89	c ................................ | *********.           70228000 | 1719798      3510556          4608.0     15240.5    116900462
4703.92/1178.94	c ................................ | *********.           71307000 | 1745695      3570970          4704.1     15158.5    116900462
4799.73/1202.98	c CPU limit (4800) reached
4799.73/1202.98	c Time = 4800.06 seconds
4799.73/1202.98	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3258658-1304052151/watcher-3258658-1304052151 -o /tmp/evaluation-result-3258658-1304052151/solver-3258658-1304052151 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3258658-1304052151.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.72 7.99 7.93 5/181 30605
/proc/meminfo: memFree=15021712/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=9760 CPUtime=0 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) R 30603 30605 30368 0 -1 4202496 490 0 0 0 0 0 0 0 25 0 1 0 516814766 9994240 433 33554432000 4194304 4338100 140735213848496 18446744073709551615 274473628357 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/30605/statm: 2440 434 147 36 0 345 0

[startup+0.0870441 s]
/proc/loadavg: 7.72 7.99 7.93 5/181 30605
/proc/meminfo: memFree=15021712/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=26168 CPUtime=0.08 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) R 30603 30605 30368 0 -1 4202496 4504 0 0 0 8 0 0 0 25 0 1 0 516814766 26796032 4447 33554432000 4194304 4338100 140735213848496 18446744073709551615 274473628357 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/30605/statm: 6542 4449 148 36 0 4447 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 26168

[startup+0.101037 s]
/proc/loadavg: 7.72 7.99 7.93 5/181 30605
/proc/meminfo: memFree=15021712/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=28808 CPUtime=0.09 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) R 30603 30605 30368 0 -1 4202496 5195 0 0 0 9 0 0 0 25 0 1 0 516814766 29499392 5138 33554432000 4194304 4338100 140735213848496 18446744073709551615 274473628357 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/30605/statm: 7202 5139 148 36 0 5107 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 28808

[startup+0.301004 s]
/proc/loadavg: 7.72 7.99 7.93 5/181 30605
/proc/meminfo: memFree=15021712/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=67444 CPUtime=0.29 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) R 30603 30605 30368 0 -1 4202496 14901 0 0 0 28 1 0 0 25 0 1 0 516814766 69062656 14843 33554432000 4194304 4338100 140735213848496 18446744073709551615 4206976 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/30605/statm: 16861 14843 153 36 0 14766 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 67444

[startup+0.700934 s]
/proc/loadavg: 7.72 7.99 7.93 5/181 30605
/proc/meminfo: memFree=15021712/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=76772 CPUtime=0.69 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) R 30603 30605 30368 0 -1 4202496 18479 0 0 0 68 1 0 0 25 0 1 0 516814766 78614528 17227 33554432000 4194304 4338100 140735213848496 18446744073709551615 274473623536 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/30605/statm: 19193 17228 156 36 0 17098 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 76772

[startup+1.50079 s]
/proc/loadavg: 7.72 7.99 7.93 6/182 30606
/proc/meminfo: memFree=14896720/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=180476 CPUtime=1.49 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) R 30603 30605 30368 0 -1 4202496 44980 0 0 0 145 4 0 0 25 0 1 0 516814766 184807424 43131 33554432000 4194304 4338100 140735213848496 18446744073709551615 274473616300 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/30605/statm: 45119 43131 164 36 0 43024 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 180476

[startup+3.10051 s]
/proc/loadavg: 7.72 7.99 7.93 9/186 30610
/proc/meminfo: memFree=14753220/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=410272 CPUtime=7.56 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) S 30603 30605 30368 0 -1 4202496 58196 0 0 0 741 15 0 0 25 0 5 0 516814766 420118528 55749 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 102568 55749 176 36 0 100473 0
[pid=30605/tid=30607] ppid=30603 vsize=410272 CPUtime=1.49 cores=0,2,4,6
/proc/30605/task/30607/stat : 30607 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 2800 0 0 0 149 0 0 0 25 0 5 0 516814927 420118528 55749 33554432000 4194304 4338100 140735213848496 18446744073709551615 4285730 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30605/tid=30608] ppid=30603 vsize=410272 CPUtime=1.48 cores=0,2,4,6
/proc/30605/task/30608/stat : 30608 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 1373 0 0 0 144 4 0 0 25 0 5 0 516814927 420118528 55749 33554432000 4194304 4338100 140735213848496 18446744073709551615 4206915 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30605/tid=30609] ppid=30603 vsize=410272 CPUtime=1.49 cores=0,2,4,6
/proc/30605/task/30609/stat : 30609 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 3456 0 0 0 148 1 0 0 25 0 5 0 516814927 420118528 55750 33554432000 4194304 4338100 140735213848496 18446744073709551615 274473362484 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30605/tid=30610] ppid=30603 vsize=410272 CPUtime=1.48 cores=0,2,4,6
/proc/30605/task/30610/stat : 30610 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 1475 0 0 0 144 4 0 0 25 0 5 0 516814927 420118528 55750 33554432000 4194304 4338100 140735213848496 18446744073709551615 274473362866 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 7.56
Current children cumulated vsize (KiB) 410272

[startup+6.30095 s]
/proc/loadavg: 7.75 7.99 7.93 9/186 30610
/proc/meminfo: memFree=14609892/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=463844 CPUtime=20.35 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) S 30603 30605 30368 0 -1 4202496 72209 0 0 0 2000 35 0 0 25 0 5 0 516814766 474976256 69762 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 115961 69762 176 36 0 113866 0
[pid=30605/tid=30607] ppid=30603 vsize=463844 CPUtime=4.69 cores=0,2,4,6
/proc/30605/task/30607/stat : 30607 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 7136 0 0 0 468 1 0 0 25 0 5 0 516814927 474976256 69762 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30605/tid=30608] ppid=30603 vsize=463844 CPUtime=4.69 cores=0,2,4,6
/proc/30605/task/30608/stat : 30608 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 3484 0 0 0 456 13 0 0 25 0 5 0 516814927 474976256 69762 33554432000 4194304 4338100 140735213848496 18446744073709551615 274473362479 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30605/tid=30609] ppid=30603 vsize=463844 CPUtime=4.69 cores=0,2,4,6
/proc/30605/task/30609/stat : 30609 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 8814 0 0 0 467 2 0 0 25 0 5 0 516814927 474976256 69762 33554432000 4194304 4338100 140735213848496 18446744073709551615 274473362657 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30605/tid=30610] ppid=30603 vsize=463844 CPUtime=4.66 cores=0,2,4,6
/proc/30605/task/30610/stat : 30610 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 3683 0 0 0 454 12 0 0 25 0 5 0 516814927 474976256 69762 33554432000 4194304 4338100 140735213848496 18446744073709551615 4253954 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 20.35
Current children cumulated vsize (KiB) 463844

[startup+12.7008 s]
/proc/loadavg: 7.77 7.99 7.93 9/186 30610
/proc/meminfo: memFree=14355452/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=558528 CPUtime=45.92 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) S 30603 30605 30368 0 -1 4202496 96645 0 0 0 4509 83 0 0 25 0 5 0 516814766 571932672 94198 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 139632 94198 176 36 0 137537 0
[pid=30605/tid=30607] ppid=30603 vsize=558528 CPUtime=11.08 cores=0,2,4,6
/proc/30605/task/30607/stat : 30607 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 14307 0 0 0 1104 4 0 0 25 0 5 0 516814927 571932672 94198 33554432000 4194304 4338100 140735213848496 18446744073709551615 274473362497 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30605/tid=30608] ppid=30603 vsize=558528 CPUtime=11.08 cores=0,2,4,6
/proc/30605/task/30608/stat : 30608 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 7319 0 0 0 1074 34 0 0 25 0 5 0 516814927 571932672 94198 33554432000 4194304 4338100 140735213848496 18446744073709551615 4256084 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30605/tid=30609] ppid=30603 vsize=558528 CPUtime=11.08 cores=0,2,4,6
/proc/30605/task/30609/stat : 30609 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 18189 0 0 0 1105 3 0 0 25 0 5 0 516814927 571932672 94198 33554432000 4194304 4338100 140735213848496 18446744073709551615 274473362675 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30605/tid=30610] ppid=30603 vsize=558528 CPUtime=11.03 cores=0,2,4,6
/proc/30605/task/30610/stat : 30610 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 7738 0 0 0 1069 34 0 0 25 0 5 0 516814927 571932672 94198 33554432000 4194304 4338100 140735213848496 18446744073709551615 4240540 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 45.92
Current children cumulated vsize (KiB) 558528

[startup+25.5006 s]
/proc/loadavg: 7.82 7.99 7.93 9/186 30611
/proc/meminfo: memFree=13947644/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=732676 CPUtime=97.04 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) S 30603 30605 30368 0 -1 4202496 139025 0 0 0 9534 170 0 0 25 0 5 0 516814766 750260224 136578 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 183169 136579 176 36 0 181074 0
[pid=30605/tid=30607] ppid=30603 vsize=732676 CPUtime=23.89 cores=0,2,4,6
/proc/30605/task/30607/stat : 30607 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 28607 0 0 0 2380 9 0 0 25 0 5 0 516814927 750260224 136579 33554432000 4194304 4338100 140735213848496 18446744073709551615 4256737 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30605/tid=30608] ppid=30603 vsize=732676 CPUtime=23.88 cores=0,2,4,6
/proc/30605/task/30608/stat : 30608 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 13244 0 0 0 2314 74 0 0 25 0 5 0 516814927 750260224 136579 33554432000 4194304 4338100 140735213848496 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30605/tid=30609] ppid=30603 vsize=732676 CPUtime=23.89 cores=0,2,4,6
/proc/30605/task/30609/stat : 30609 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 34062 0 0 0 2383 6 0 0 25 0 5 0 516814927 750260224 136579 33554432000 4194304 4338100 140735213848496 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30605/tid=30610] ppid=30603 vsize=732676 CPUtime=23.75 cores=0,2,4,6
/proc/30605/task/30610/stat : 30610 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 14021 0 0 0 2301 74 0 0 25 0 5 0 516814927 750260224 136579 33554432000 4194304 4338100 140735213848496 18446744073709551615 4239484 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 97.04
Current children cumulated vsize (KiB) 732676

[startup+51.1011 s]
/proc/loadavg: 7.88 7.99 7.93 9/186 30611
/proc/meminfo: memFree=13264584/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=997348 CPUtime=199.3 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) S 30603 30605 30368 0 -1 4202496 205880 0 0 0 19592 338 0 0 25 0 5 0 516814766 1021284352 203433 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 249337 203433 176 36 0 247242 0
[pid=30605/tid=30607] ppid=30603 vsize=997348 CPUtime=49.48 cores=0,2,4,6
/proc/30605/task/30607/stat : 30607 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 55110 0 0 0 4930 18 0 0 25 0 5 0 516814927 1021284352 203433 33554432000 4194304 4338100 140735213848496 18446744073709551615 4240496 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30605/tid=30608] ppid=30603 vsize=997348 CPUtime=49.37 cores=0,2,4,6
/proc/30605/task/30608/stat : 30608 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 20758 0 0 0 4787 150 0 0 25 0 5 0 516814927 1021284352 203433 33554432000 4194304 4338100 140735213848496 18446744073709551615 274473362484 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30605/tid=30609] ppid=30603 vsize=997348 CPUtime=49.48 cores=0,2,4,6
/proc/30605/task/30609/stat : 30609 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 57607 0 0 0 4936 12 0 0 25 0 5 0 516814927 1021284352 203433 33554432000 4194304 4338100 140735213848496 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30605/tid=30610] ppid=30603 vsize=997348 CPUtime=49.32 cores=0,2,4,6
/proc/30605/task/30610/stat : 30610 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 23313 0 0 0 4782 150 0 0 25 0 5 0 516814927 1021284352 203433 33554432000 4194304 4338100 140735213848496 18446744073709551615 274473363182 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 199.3
Current children cumulated vsize (KiB) 997348

[startup+102.306 s]
/proc/loadavg: 7.95 7.99 7.93 9/186 30613
/proc/meminfo: memFree=12393844/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=1301484 CPUtime=403.84 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) S 30603 30605 30368 0 -1 4202496 278543 0 0 0 39740 644 0 0 25 0 5 0 516814766 1332719616 273241 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 325371 273241 176 36 0 323276 0
[pid=30605/tid=30607] ppid=30603 vsize=1301484 CPUtime=100.69 cores=0,2,4,6
/proc/30605/task/30607/stat : 30607 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 87645 0 0 0 10039 30 0 0 25 0 5 0 516814927 1332719616 273241 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30605/tid=30608] ppid=30603 vsize=1301484 CPUtime=100.3 cores=0,2,4,6
/proc/30605/task/30608/stat : 30608 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 28024 0 0 0 9734 296 0 0 25 0 5 0 516814927 1332719616 273241 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30605/tid=30609] ppid=30603 vsize=1301484 CPUtime=100.68 cores=0,2,4,6
/proc/30605/task/30609/stat : 30609 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 82677 0 0 0 10050 18 0 0 25 0 5 0 516814927 1332719616 273241 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30605/tid=30610] ppid=30603 vsize=1301484 CPUtime=100.52 cores=0,2,4,6
/proc/30605/task/30610/stat : 30610 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 31105 0 0 0 9759 293 0 0 25 0 5 0 516814927 1332719616 273241 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254304 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 403.84
Current children cumulated vsize (KiB) 1301484

[startup+162.3 s]
/proc/loadavg: 8.05 8.01 7.93 9/186 30614
/proc/meminfo: memFree=11614868/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=1573620 CPUtime=643.49 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) S 30603 30605 30368 0 -1 4202496 346830 0 0 0 63408 941 0 0 25 0 5 0 516814766 1611386880 341528 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 393405 341528 176 36 0 391310 0
[pid=30605/tid=30607] ppid=30603 vsize=1573620 CPUtime=160.68 cores=0,2,4,6
/proc/30605/task/30607/stat : 30607 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 120509 0 0 0 16025 43 0 0 25 0 5 0 516814927 1611386880 341528 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254213 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30605/tid=30608] ppid=30603 vsize=1573620 CPUtime=160.16 cores=0,2,4,6
/proc/30605/task/30608/stat : 30608 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 30507 0 0 0 15585 431 0 0 25 0 5 0 516814927 1611386880 341528 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30605/tid=30609] ppid=30603 vsize=1573620 CPUtime=160.67 cores=0,2,4,6
/proc/30605/task/30609/stat : 30609 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 111723 0 0 0 16041 26 0 0 25 0 5 0 516814927 1611386880 341528 33554432000 4194304 4338100 140735213848496 18446744073709551615 4284167 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30605/tid=30610] ppid=30603 vsize=1573620 CPUtime=160.33 cores=0,2,4,6
/proc/30605/task/30610/stat : 30610 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 34999 0 0 0 15600 433 0 0 25 0 5 0 516814927 1611386880 341528 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254239 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 643.49
Current children cumulated vsize (KiB) 1573620

[startup+222.301 s]
/proc/loadavg: 8.15 8.03 7.94 9/186 30616
/proc/meminfo: memFree=11148120/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=1819316 CPUtime=883.1 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) S 30603 30605 30368 0 -1 4202496 408294 0 0 0 87101 1209 0 0 25 0 5 0 516814766 1862979584 402992 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 454829 402992 176 36 0 452734 0
[pid=30605/tid=30607] ppid=30603 vsize=1819316 CPUtime=220.66 cores=0,2,4,6
/proc/30605/task/30607/stat : 30607 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 148636 0 0 0 22012 54 0 0 25 0 5 0 516814927 1862979584 402992 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0

################
# More data... #
################

/proc/30605/task/30610/stat : 30610 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 51219 0 0 0 68518 1374 0 0 25 0 5 0 516814927 3269431296 746457 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2800.39
Current children cumulated vsize (KiB) 3192804

[startup+762.307 s]
/proc/loadavg: 8.00 8.04 7.99 9/186 30630
/proc/meminfo: memFree=7633448/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=3333564 CPUtime=3040.1 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) S 30603 30605 30368 0 -1 4202496 788105 0 0 0 300871 3139 0 0 25 0 5 0 516814766 3413569536 782803 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 833391 782803 176 36 0 831296 0
[pid=30605/tid=30607] ppid=30603 vsize=3333564 CPUtime=759.22 cores=0,2,4,6
/proc/30605/task/30607/stat : 30607 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 365773 0 0 0 75775 147 0 0 25 0 5 0 516814927 3413569536 782803 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30605/tid=30608] ppid=30603 vsize=3333564 CPUtime=759.68 cores=0,2,4,6
/proc/30605/task/30608/stat : 30608 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 40463 0 0 0 74519 1449 0 0 25 0 5 0 516814927 3413569536 782803 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30605/tid=30609] ppid=30603 vsize=3333564 CPUtime=760.63 cores=0,2,4,6
/proc/30605/task/30609/stat : 30609 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 281558 0 0 0 75994 69 0 0 25 0 5 0 516814927 3413569536 782803 33554432000 4194304 4338100 140735213848496 18446744073709551615 4256749 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30605/tid=30610] ppid=30603 vsize=3333564 CPUtime=758.93 cores=0,2,4,6
/proc/30605/task/30610/stat : 30610 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 51219 0 0 0 74426 1467 0 0 25 0 5 0 516814927 3413569536 782803 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3040.1
Current children cumulated vsize (KiB) 3333564

[startup+822.3 s]
/proc/loadavg: 8.15 8.07 8.00 9/186 30632
/proc/meminfo: memFree=7414176/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=3508732 CPUtime=3279.76 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) S 30603 30605 30368 0 -1 4202496 830742 0 0 0 324635 3341 0 0 25 0 5 0 516814766 3592941568 825440 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 877183 825440 176 36 0 875088 0
[pid=30605/tid=30607] ppid=30603 vsize=3508732 CPUtime=818.92 cores=0,2,4,6
/proc/30605/task/30607/stat : 30607 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 390174 0 0 0 81733 159 0 0 25 0 5 0 516814927 3592941568 825440 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30605/tid=30608] ppid=30603 vsize=3508732 CPUtime=819.66 cores=0,2,4,6
/proc/30605/task/30608/stat : 30608 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 40463 0 0 0 80423 1543 0 0 25 0 5 0 516814927 3592941568 825440 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30605/tid=30609] ppid=30603 vsize=3508732 CPUtime=820.63 cores=0,2,4,6
/proc/30605/task/30609/stat : 30609 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 299794 0 0 0 81989 74 0 0 25 0 5 0 516814927 3592941568 825440 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30605/tid=30610] ppid=30603 vsize=3508732 CPUtime=818.9 cores=0,2,4,6
/proc/30605/task/30610/stat : 30610 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 51219 0 0 0 80333 1557 0 0 25 0 5 0 516814927 3592941568 825440 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3279.76
Current children cumulated vsize (KiB) 3508732

[startup+882.301 s]
/proc/loadavg: 8.24 8.10 8.01 9/186 30633
/proc/meminfo: memFree=7267168/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=3675108 CPUtime=3519.35 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) S 30603 30605 30368 0 -1 4202496 856906 0 0 0 348394 3541 0 0 25 0 5 0 516814766 3763310592 851604 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 918777 851604 176 36 0 916682 0
[pid=30605/tid=30607] ppid=30603 vsize=3675108 CPUtime=878.89 cores=0,2,4,6
/proc/30605/task/30607/stat : 30607 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 402728 0 0 0 87723 166 0 0 25 0 5 0 516814927 3763310592 851604 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30605/tid=30608] ppid=30603 vsize=3675108 CPUtime=879.34 cores=0,2,4,6
/proc/30605/task/30608/stat : 30608 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 42372 0 0 0 86295 1639 0 0 25 0 5 0 516814927 3763310592 851604 33554432000 4194304 4338100 140735213848496 18446744073709551615 274473362448 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30605/tid=30609] ppid=30603 vsize=3675108 CPUtime=880.61 cores=0,2,4,6
/proc/30605/task/30609/stat : 30609 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 308584 0 0 0 87984 77 0 0 25 0 5 0 516814927 3763310592 851604 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254362 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30605/tid=30610] ppid=30603 vsize=3675108 CPUtime=878.88 cores=0,2,4,6
/proc/30605/task/30610/stat : 30610 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 54130 0 0 0 86235 1653 0 0 25 0 5 0 516814927 3763310592 851604 33554432000 4194304 4338100 140735213848496 18446744073709551615 4258020 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3519.35
Current children cumulated vsize (KiB) 3675108

[startup+942.3 s]
/proc/loadavg: 8.09 8.08 8.01 9/186 30635
/proc/meminfo: memFree=7006104/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=3831508 CPUtime=3759 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) S 30603 30605 30368 0 -1 4202496 897206 0 0 0 372168 3732 0 0 25 0 5 0 516814766 3923464192 891904 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 957877 891904 176 36 0 955782 0
[pid=30605/tid=30607] ppid=30603 vsize=3831508 CPUtime=938.88 cores=0,2,4,6
/proc/30605/task/30607/stat : 30607 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 424734 0 0 0 93713 175 0 0 25 0 5 0 516814927 3923464192 891904 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30605/tid=30608] ppid=30603 vsize=3831508 CPUtime=939.01 cores=0,2,4,6
/proc/30605/task/30608/stat : 30608 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 42785 0 0 0 92171 1730 0 0 25 0 5 0 516814927 3923464192 891904 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30605/tid=30609] ppid=30603 vsize=3831508 CPUtime=940.6 cores=0,2,4,6
/proc/30605/task/30609/stat : 30609 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 326088 0 0 0 93979 81 0 0 25 0 5 0 516814927 3923464192 891904 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30605/tid=30610] ppid=30603 vsize=3831508 CPUtime=938.87 cores=0,2,4,6
/proc/30605/task/30610/stat : 30610 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 54507 0 0 0 92149 1738 0 0 25 0 5 0 516814927 3923464192 891904 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3759
Current children cumulated vsize (KiB) 3831508

[startup+1002.3 s]
/proc/loadavg: 8.10 8.08 8.01 9/186 30637
/proc/meminfo: memFree=6648456/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=4034828 CPUtime=3998.66 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) S 30603 30605 30368 0 -1 4202496 947796 0 0 0 395937 3929 0 0 25 0 5 0 516814766 4131663872 942494 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 1008707 942494 176 36 0 1006612 0
[pid=30605/tid=30607] ppid=30603 vsize=4034828 CPUtime=998.88 cores=0,2,4,6
/proc/30605/task/30607/stat : 30607 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 451375 0 0 0 99703 185 0 0 25 0 5 0 516814927 4131663872 942494 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30605/tid=30608] ppid=30603 vsize=4034828 CPUtime=998.67 cores=0,2,4,6
/proc/30605/task/30608/stat : 30608 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 42785 0 0 0 98047 1820 0 0 25 0 5 0 516814927 4131663872 942494 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30605/tid=30609] ppid=30603 vsize=4034828 CPUtime=1000.6 cores=0,2,4,6
/proc/30605/task/30609/stat : 30609 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 350037 0 0 0 99972 88 0 0 25 0 5 0 516814927 4131663872 942494 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485780262 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30605/tid=30610] ppid=30603 vsize=4034828 CPUtime=998.87 cores=0,2,4,6
/proc/30605/task/30610/stat : 30610 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 54507 0 0 0 98058 1829 0 0 25 0 5 0 516814927 4131663872 942494 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3998.66
Current children cumulated vsize (KiB) 4034828

[startup+1062.3 s]
/proc/loadavg: 8.10 8.08 8.01 9/186 30638
/proc/meminfo: memFree=6345508/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=4112876 CPUtime=4238.28 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) S 30603 30605 30368 0 -1 4202496 960839 0 0 0 419714 4114 0 0 25 0 5 0 516814766 4211585024 955537 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 1028219 955537 176 36 0 1026124 0
[pid=30605/tid=30607] ppid=30603 vsize=4112876 CPUtime=1058.86 cores=0,2,4,6
/proc/30605/task/30607/stat : 30607 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 454468 0 0 0 105697 189 0 0 25 0 5 0 516814927 4211585024 955537 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30605/tid=30608] ppid=30603 vsize=4112876 CPUtime=1058.38 cores=0,2,4,6
/proc/30605/task/30608/stat : 30608 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 48145 0 0 0 103928 1910 0 0 25 0 5 0 516814927 4211585024 955537 33554432000 4194304 4338100 140735213848496 18446744073709551615 4253954 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30605/tid=30609] ppid=30603 vsize=4112876 CPUtime=1060.59 cores=0,2,4,6
/proc/30605/task/30609/stat : 30609 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 350522 0 0 0 105971 88 0 0 25 0 5 0 516814927 4211585024 955537 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30605/tid=30610] ppid=30603 vsize=4112876 CPUtime=1058.8 cores=0,2,4,6
/proc/30605/task/30610/stat : 30610 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 58612 0 0 0 103961 1919 0 0 25 0 5 0 516814927 4211585024 955537 33554432000 4194304 4338100 140735213848496 18446744073709551615 4240513 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4238.28
Current children cumulated vsize (KiB) 4112876

[startup+1122.3 s]
/proc/loadavg: 8.09 8.08 8.01 9/186 30649
/proc/meminfo: memFree=6071636/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=4209844 CPUtime=4477.87 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) S 30603 30605 30368 0 -1 4202496 987521 0 0 0 443467 4320 0 0 25 0 5 0 516814766 4310880256 982219 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 1052461 982219 176 36 0 1050366 0
[pid=30605/tid=30607] ppid=30603 vsize=4209844 CPUtime=1118.83 cores=0,2,4,6
/proc/30605/task/30607/stat : 30607 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 472736 0 0 0 111684 199 0 0 25 0 5 0 516814927 4310880256 982219 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30605/tid=30608] ppid=30603 vsize=4209844 CPUtime=1118.37 cores=0,2,4,6
/proc/30605/task/30608/stat : 30608 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 48986 0 0 0 109833 2004 0 0 25 0 5 0 516814927 4310880256 982219 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30605/tid=30609] ppid=30603 vsize=4209844 CPUtime=1120.57 cores=0,2,4,6
/proc/30605/task/30609/stat : 30609 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 356687 0 0 0 111967 90 0 0 25 0 5 0 516814927 4310880256 982219 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30605/tid=30610] ppid=30603 vsize=4209844 CPUtime=1118.47 cores=0,2,4,6
/proc/30605/task/30610/stat : 30610 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 60020 0 0 0 109827 2020 0 0 25 0 5 0 516814927 4310880256 982219 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4477.87
Current children cumulated vsize (KiB) 4209844

[startup+1182.3 s]
/proc/loadavg: 8.03 8.06 8.00 9/186 30650
/proc/meminfo: memFree=5821152/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=4309940 CPUtime=4717.5 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) S 30603 30605 30368 0 -1 4202496 1013826 0 0 0 467220 4530 0 0 25 0 5 0 516814766 4413378560 1008524 33554432000 4194304 4338100 140735213848496 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 1077485 1008524 176 36 0 1075390 0
[pid=30605/tid=30607] ppid=30603 vsize=4309940 CPUtime=1178.81 cores=0,2,4,6
/proc/30605/task/30607/stat : 30607 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 487064 0 0 0 117675 206 0 0 25 0 5 0 516814927 4413378560 1008524 33554432000 4194304 4338100 140735213848496 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30605/tid=30608] ppid=30603 vsize=4309940 CPUtime=1178.37 cores=0,2,4,6
/proc/30605/task/30608/stat : 30608 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 49621 0 0 0 115734 2103 0 0 25 0 5 0 516814927 4413378560 1008524 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30605/tid=30609] ppid=30603 vsize=4309940 CPUtime=1180.56 cores=0,2,4,6
/proc/30605/task/30609/stat : 30609 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 367558 0 0 0 117963 93 0 0 25 0 5 0 516814927 4413378560 1008524 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30605/tid=30610] ppid=30603 vsize=4309940 CPUtime=1178.13 cores=0,2,4,6
/proc/30605/task/30610/stat : 30610 (ttsth-5-4) R 30603 30605 30368 0 -1 4202560 60491 0 0 0 115692 2121 0 0 25 0 5 0 516814927 4413378560 1008524 33554432000 4194304 4338100 140735213848496 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4717.5
Current children cumulated vsize (KiB) 4309940



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203 s]
/proc/loadavg: 7.70 7.99 7.98 2/174 30669
/proc/meminfo: memFree=11460504/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=0 CPUtime=4800.08 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) Z 30603 30605 30368 0 -1 4203548 1023513 0 0 0 475403 4605 0 0 18 0 2 0 516814766 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 0 0 0 0 0 0 0
[pid=30605/tid=30608] ppid=30603 vsize=0 CPUtime=1199.05 cores=0,2,4,6
/proc/30605/task/30608/stat : 30608 (ttsth-5-4) R 30603 30605 30368 0 -1 4203588 52678 0 0 0 117766 2139 0 0 25 0 2 0 516814927 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.08
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.11 s]
/proc/loadavg: 7.70 7.99 7.98 2/174 30669
/proc/meminfo: memFree=11460504/32950928 swapFree=67111528/67111528
[pid=30605] ppid=30603 vsize=0 CPUtime=4800.19 cores=0,2,4,6
/proc/30605/stat : 30605 (ttsth-5-4) Z 30603 30605 30368 0 -1 4203548 1023513 0 0 0 475403 4616 0 0 18 0 2 0 516814766 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30605/statm: 0 0 0 0 0 0 0
[pid=30605/tid=30608] ppid=30603 vsize=0 CPUtime=1199.15 cores=0,2,4,6
/proc/30605/task/30608/stat : 30608 (ttsth-5-4) R 30603 30605 30368 0 -1 4203588 52678 0 0 0 117766 2149 0 0 25 0 2 0 516814927 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.19
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1203.14
CPU time (s): 4800.23
CPU user time (s): 4754.04
CPU system time (s): 46.189
CPU usage (%): 398.975
Max. virtual memory (cumulated for all children) (KiB): 4514740

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4754.04
system time used= 46.189
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1023513
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4320
involuntary context switches= 29505

runsolver used 2.11168 second user time and 4.36634 second system time

The end

Launcher Data 

Begin job on node125 at 2011-04-29 06:42:31
IDJOB=3258658
IDBENCH=71408
IDSOLVER=1654
FILE ID=node125/3258658-1304052151
RUNJOBID= node125-1304049739-30386
PBS_JOBID= 13170412
Free space on /tmp= 72760 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT09/CRAFTED/rbsat/random/forced/rbsat-v2640c305320gyes5.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3258658-1304052151/watcher-3258658-1304052151 -o /tmp/evaluation-result-3258658-1304052151/solver-3258658-1304052151 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3258658-1304052151.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f892a2bda6fe6aec6f2fcb4e1c541df8
RANDOM SEED=166370810

node125.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5266.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      15022116 kB
Buffers:       2218204 kB
Cached:       14975944 kB
SwapCached:          0 kB
Active:        9184936 kB
Inactive:      8213220 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      15022116 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            5444 kB
Writeback:           0 kB
AnonPages:      203004 kB
Mapped:          15212 kB
Slab:           468696 kB
PageTables:       4924 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   394996 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72760 MiB
End job on node125 at 2011-04-29 07:02:36