Trace number 3259274

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.31 1205.38

General information on the benchmark

NameCRAFTED/rbsat/crafted/forced/
rbsat-v945c61409gyes7.cnf
MD5SUM6200c727c8a5a324974090a1f9a6d7ab
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark2.03669
Satisfiable
(Un)Satisfiability was proved
Number of variables945
Number of clauses61409
Sum of the clauses size123673
Maximum clause length21
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 261364
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 545

Solver Data 

0.02/0.04	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259274-1304054569.cnf 
0.02/0.04	c Using SATTIMEOUT of 4800
0.02/0.04	c Using SATRAM of 15500
0.02/0.04	c Size of subset node = 32
0.02/0.04	c Variables = 945
0.02/0.04	c Clauses = 61409
0.02/0.04	c Literals = 123673
0.02/0.04	c Reduced size:
0.02/0.04	c Variables = 945
0.02/0.04	c Clauses = 61409
0.02/0.04	c Literals = 123673
0.02/0.04	c a2
0.02/0.07	c a1
0.09/0.14	c Independent sub-problems = 1
0.09/0.14	c Sub-problem with 945 variables and 61409 clauses
0.09/0.14	c using permutation method 4
0.09/0.14	c count = 1
0.09/0.14	c Ordering finished 0.130 seconds
0.09/0.14	c mapped 0.130 seconds
0.09/0.14	c permuted 0.130 seconds
0.09/0.14	c about to sort
0.09/0.15	c sorted
0.20/0.27	c clauses now 61409
0.20/0.27	c init finished 0.150 seconds
0.20/0.27	c tree built 0.260 seconds
0.20/0.27	c 
0.20/0.27	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.20/0.27	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.20/0.27	c ................................ | ..........                    |                                  0.3         0.0             
0.20/0.27	c ................................ | ..........                  1 |              8                   0.3         3.8             
0.20/0.27	c ................................ | ..........                  2 |              8                   0.3         7.7             
0.20/0.27	c ................................ | ..........                  3 |              8                   0.3        11.5             
0.20/0.27	c ................................ | ..........                  4 |              9                   0.3        15.4             
0.20/0.27	c ................................ | ..........                  5 |              9                   0.3        19.2             
0.20/0.28	c ................................ | ..........               1000 | 15           142                 0.3      3571.4         8346
95.84/24.34	c ................................ | ..........            3296000 | 89102        482212             96.0     34326.2     21413148
191.80/48.46	c ................................ | ..........            5726000 | 159633       827830            192.0     29819.8     34983853
287.75/72.56	c ................................ | ..........            7732000 | 221460       1107240           288.0     26847.2     42690728
383.71/96.67	c ................................ | ..........            9759000 | 283628       1384205           384.0     25414.1     44853478
479.66/120.78	c ................................ | *.........           11938000 | 350609       1655482           480.0     24870.8     52354904
575.68/144.88	c ................................ | *.........           13755000 | 411707       1872478           576.0     23879.0     56008726
672.02/169.00	c ................................ | *.........           15747000 | 465666       2141932           672.0     23432.0     64817328
768.02/193.10	c ................................ | *.........           17628000 | 514049       2391910           768.0     22952.5     71307277
863.72/217.17	c ................................ | *.........           19519000 | 562195       2632790           864.0     22590.7     79215194
959.74/241.27	c ................................ | **........           21205000 | 608987       2887110           960.0     22087.6     89618148
1055.76/265.37	c ................................ | **........           22711000 | 653904       3112855          1056.0     21505.8     96406448
1151.76/289.46	c ................................ | **........           24126000 | 694883       3316166          1152.0     20942.7     97885441
1247.81/313.56	c ................................ | **........           25380000 | 734060       3507380          1248.0     20336.2    100178064
1343.81/337.66	c ................................ | **........           26572000 | 772353       3690264          1344.0     19770.1    106002470
1439.79/361.75	c ................................ | ***.......           27920000 | 809250       3912393          1440.0     19388.8    112247334
1535.74/385.87	c ................................ | ***.......           29274000 | 848364       4154997          1536.0     19058.1    120240561
1631.72/409.98	c ................................ | ***.......           30504000 | 890003       4365034          1632.0     18690.7    128596705
1727.71/434.08	c ................................ | ***.......           31759000 | 928439       4582853          1728.0     18378.5    135406456
1823.71/458.17	c ................................ | ***.......           32948000 | 965249       4767869          1824.0     18063.5    140395423
1919.99/482.32	c ................................ | ****......           34054000 | 998617       4947458          1920.1     17735.7    140395423
2015.63/506.39	c ................................ | ****......           35124000 | 1032139      5154210          2016.0     17422.6    140395423
2111.69/530.48	c ................................ | ****......           36108000 | 1060993      5333482          2112.1     17096.2    140395423
2207.69/554.59	c ................................ | ****......           37251000 | 1099228      5499889          2208.1     16870.4    140807371
2303.65/578.69	c ................................ | ****......           38375000 | 1137434      5652343          2304.0     16655.7    141113360
2399.63/602.80	c ................................ | *****.....           39457000 | 1173577      5802426          2400.0     16440.2    141113360
2496.03/626.91	c ................................ | *****.....           40468000 | 1207344      5963563          2496.1     16212.7    141995365
2592.08/651.00	c ................................ | *****.....           41557000 | 1241852      6141821          2592.1     16032.3    142780357
2687.77/675.07	c ................................ | *****.....           42642000 | 1276862      6298644          2688.1     15863.4    144588826
2783.83/699.15	c ................................ | *****.....           43734000 | 1308786      6451427          2784.0     15708.9    149548957
2879.86/723.24	c ................................ | ******....           45096000 | 1352041      6601599          2880.1     15658.1    149548957
2975.85/747.34	c ................................ | ******....           46346000 | 1396273      6753006          2976.0     15573.1    149961211
3071.83/771.45	c ................................ | ******....           47762000 | 1438027      6927420          3072.1     15547.3    151321223
3167.74/795.56	c ................................ | ******....           49189000 | 1477412      7124611          3168.0     15526.8    153247110
3263.79/819.66	c ................................ | ******....           50528000 | 1514740      7299163          3264.0     15480.2    156621436
3359.79/843.76	c ................................ | *******...           51676000 | 1549946      7467517          3360.0     15379.6    158366027
3455.87/867.84	c ................................ | *******...           52769000 | 1587959      7628932          3456.0     15268.7    158366027
3551.88/891.94	c ................................ | *******...           53780000 | 1622168      7793014          3552.1     15140.5    158794346
3647.94/916.03	c ................................ | *******...           54913000 | 1654281      7959788          3648.1     15052.6    162143647
3744.00/940.10	c ................................ | *******...           55893000 | 1690591      8087825          3744.0     14928.7    164398863
3840.03/964.21	c ................................ | ********..           56906000 | 1725543      8232052          3840.1     14819.0    169707582
3936.03/988.31	c ................................ | ********..           57935000 | 1757244      8396817          3936.1     14718.9    176042273
4032.06/1012.40	c ................................ | ********..           58888000 | 1792070      8543672          4032.1     14604.9    177386013
4127.65/1036.49	c ................................ | ********..           59814000 | 1827348      8679149          4128.0     14489.7    179340675
4223.96/1060.63	c ................................ | ********..           60801000 | 1858930      8815388          4224.1     14393.8    179737427
4320.03/1084.70	c ................................ | *********.           61727000 | 1890066      8969069          4320.1     14288.5    182379630
4416.05/1108.80	c ................................ | *********.           62757000 | 1923322      9125432          4416.1     14211.1    185028511
4511.65/1132.89	c ................................ | *********.           63944000 | 1955491      9291829          4512.0     14171.9    185040158
4607.75/1156.97	c ................................ | *********.           64967000 | 1989019      9429440          4608.0     14098.6    185040158
4703.80/1181.06	c ................................ | *********.           65959000 | 2021635      9559425          4704.1     14021.7    185324694
4799.88/1205.15	c CPU limit (4800) reached
4799.88/1205.15	c Time = 4800.08 seconds
4799.88/1205.15	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3259274-1304054569/watcher-3259274-1304054569 -o /tmp/evaluation-result-3259274-1304054569/solver-3259274-1304054569 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259274-1304054569.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.55 7.98 7.98 5/181 30811
/proc/meminfo: memFree=15096080/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=9760 CPUtime=0 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) R 30809 30811 30368 0 -1 4202496 494 0 0 0 0 0 0 0 25 0 1 0 517056551 9994240 436 33554432000 4194304 4338100 140736759754352 18446744073709551615 274473967360 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/30811/statm: 2440 438 147 36 0 345 0

[startup+0.0270359 s]
/proc/loadavg: 7.55 7.98 7.98 5/181 30811
/proc/meminfo: memFree=15096080/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=14204 CPUtime=0.02 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) R 30809 30811 30368 0 -1 4202496 1580 0 0 0 2 0 0 0 25 0 1 0 517056551 14544896 1522 33554432000 4194304 4338100 140736759754352 18446744073709551615 274473628357 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/30811/statm: 3551 1523 148 36 0 1456 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 14204

[startup+0.101027 s]
/proc/loadavg: 7.55 7.98 7.98 5/181 30811
/proc/meminfo: memFree=15096080/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=20716 CPUtime=0.09 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) R 30809 30811 30368 0 -1 4202496 3891 0 0 0 9 0 0 0 25 0 1 0 517056551 21213184 3218 33554432000 4194304 4338100 140736759754352 18446744073709551615 274473628357 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/30811/statm: 5212 3219 155 36 0 3117 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 20716

[startup+0.300993 s]
/proc/loadavg: 7.55 7.98 7.98 5/181 30811
/proc/meminfo: memFree=15096080/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=237192 CPUtime=0.36 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 12644 0 0 0 34 2 0 0 25 0 5 0 517056551 242884608 11728 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 59298 11728 176 36 0 57203 0
Current children cumulated CPU time (s) 0.36
Current children cumulated vsize (KiB) 237192

[startup+0.700922 s]
/proc/loadavg: 7.55 7.98 7.98 5/181 30811
/proc/meminfo: memFree=15096080/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=256752 CPUtime=1.94 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 17784 0 0 0 191 3 0 0 25 0 5 0 517056551 262914048 16868 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 64188 16868 176 36 0 62093 0
Current children cumulated CPU time (s) 1.94
Current children cumulated vsize (KiB) 256752

[startup+1.50078 s]
/proc/loadavg: 7.55 7.98 7.98 9/186 30817
/proc/meminfo: memFree=14973536/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=288428 CPUtime=5.12 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 26294 0 0 0 506 6 0 0 25 0 5 0 517056551 295350272 25378 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 72107 25378 176 36 0 70012 0
[pid=30811/tid=30813] ppid=30809 vsize=288428 CPUtime=1.2 cores=0,2,4,6
/proc/30811/task/30813/stat : 30813 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 4909 0 0 0 120 0 0 0 25 0 5 0 517056578 295350272 25379 33554432000 4194304 4338100 140736759754352 18446744073709551615 4284167 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30811/tid=30814] ppid=30809 vsize=288428 CPUtime=1.19 cores=0,2,4,6
/proc/30811/task/30814/stat : 30814 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 2757 0 0 0 118 1 0 0 18 0 5 0 517056578 295350272 25379 33554432000 4194304 4338100 140736759754352 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=288428 CPUtime=1.21 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 4407 0 0 0 121 0 0 0 25 0 5 0 517056578 295350272 25379 33554432000 4194304 4338100 140736759754352 18446744073709551615 4256695 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30811/tid=30816] ppid=30809 vsize=288428 CPUtime=1.21 cores=0,2,4,6
/proc/30811/task/30816/stat : 30816 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 2275 0 0 0 120 1 0 0 18 0 5 0 517056578 295350272 25379 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.12
Current children cumulated vsize (KiB) 288428

[startup+3.1005 s]
/proc/loadavg: 7.55 7.98 7.98 9/186 30817
/proc/meminfo: memFree=14884628/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=348044 CPUtime=11.49 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 41621 0 0 0 1138 11 0 0 25 0 5 0 517056551 356397056 40705 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 87011 40705 176 36 0 84916 0
[pid=30811/tid=30813] ppid=30809 vsize=348044 CPUtime=2.8 cores=0,2,4,6
/proc/30811/task/30813/stat : 30813 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 10033 0 0 0 278 2 0 0 25 0 5 0 517056578 356397056 40705 33554432000 4194304 4338100 140736759754352 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30811/tid=30814] ppid=30809 vsize=348044 CPUtime=2.76 cores=0,2,4,6
/proc/30811/task/30814/stat : 30814 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 5717 0 0 0 274 2 0 0 18 0 5 0 517056578 356397056 40706 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=348044 CPUtime=2.82 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 9216 0 0 0 281 1 0 0 25 0 5 0 517056578 356397056 40706 33554432000 4194304 4338100 140736759754352 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30811/tid=30816] ppid=30809 vsize=348044 CPUtime=2.81 cores=0,2,4,6
/proc/30811/task/30816/stat : 30816 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 4709 0 0 0 278 3 0 0 22 0 5 0 517056578 356397056 40706 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 11.49
Current children cumulated vsize (KiB) 348044

[startup+6.30094 s]
/proc/loadavg: 7.67 7.99 7.98 9/186 30817
/proc/meminfo: memFree=14671292/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=451532 CPUtime=24.23 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 66982 0 0 0 2402 21 0 0 25 0 5 0 517056551 462368768 66066 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 112883 66066 176 36 0 110788 0
[pid=30811/tid=30813] ppid=30809 vsize=451532 CPUtime=5.98 cores=0,2,4,6
/proc/30811/task/30813/stat : 30813 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 18368 0 0 0 595 3 0 0 25 0 5 0 517056578 462368768 66066 33554432000 4194304 4338100 140736759754352 18446744073709551615 4239444 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30811/tid=30814] ppid=30809 vsize=451532 CPUtime=5.92 cores=0,2,4,6
/proc/30811/task/30814/stat : 30814 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 10152 0 0 0 588 4 0 0 18 0 5 0 517056578 462368768 66066 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=451532 CPUtime=6.01 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 18204 0 0 0 599 2 0 0 25 0 5 0 517056578 462368768 66066 33554432000 4194304 4338100 140736759754352 18446744073709551615 4256352 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30811/tid=30816] ppid=30809 vsize=451532 CPUtime=6.01 cores=0,2,4,6
/proc/30811/task/30816/stat : 30816 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 8311 0 0 0 593 8 0 0 25 0 5 0 517056578 462368768 66066 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 24.23
Current children cumulated vsize (KiB) 451532

[startup+12.7008 s]
/proc/loadavg: 7.77 8.01 7.99 9/186 30817
/proc/meminfo: memFree=14281368/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=637156 CPUtime=49.68 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 113905 0 0 0 4926 42 0 0 25 0 5 0 517056551 652447744 112989 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 159289 112989 176 36 0 157194 0
[pid=30811/tid=30813] ppid=30809 vsize=637156 CPUtime=12.34 cores=0,2,4,6
/proc/30811/task/30813/stat : 30813 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 34263 0 0 0 1227 7 0 0 19 0 5 0 517056578 652447744 112990 33554432000 4194304 4338100 140736759754352 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30811/tid=30814] ppid=30809 vsize=637156 CPUtime=12.3 cores=0,2,4,6
/proc/30811/task/30814/stat : 30814 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 17477 0 0 0 1216 14 0 0 25 0 5 0 517056578 652447744 112991 33554432000 4194304 4338100 140736759754352 18446744073709551615 4256349 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=637156 CPUtime=12.42 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 33441 0 0 0 1237 5 0 0 25 0 5 0 517056578 652447744 112992 33554432000 4194304 4338100 140736759754352 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30811/tid=30816] ppid=30809 vsize=637156 CPUtime=12.31 cores=0,2,4,6
/proc/30811/task/30816/stat : 30816 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 16779 0 0 0 1219 12 0 0 19 0 5 0 517056578 652447744 112992 33554432000 4194304 4338100 140736759754352 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 49.68
Current children cumulated vsize (KiB) 637156

[startup+25.5006 s]
/proc/loadavg: 7.89 8.03 7.99 9/186 30817
/proc/meminfo: memFree=13746808/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=934816 CPUtime=100.62 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 187870 0 0 0 9984 78 0 0 25 0 5 0 517056551 957251584 186954 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 233704 186954 176 36 0 231609 0
[pid=30811/tid=30813] ppid=30809 vsize=934816 CPUtime=25.07 cores=0,2,4,6
/proc/30811/task/30813/stat : 30813 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 58579 0 0 0 2494 13 0 0 25 0 5 0 517056578 957251584 186954 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30811/tid=30814] ppid=30809 vsize=934816 CPUtime=25.11 cores=0,2,4,6
/proc/30811/task/30814/stat : 30814 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 29995 0 0 0 2477 34 0 0 25 0 5 0 517056578 957251584 186954 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=934816 CPUtime=25.21 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 56958 0 0 0 2513 8 0 0 25 0 5 0 517056578 957251584 186954 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30811/tid=30816] ppid=30809 vsize=934816 CPUtime=24.92 cores=0,2,4,6
/proc/30811/task/30816/stat : 30816 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 30391 0 0 0 2473 19 0 0 18 0 5 0 517056578 957251584 186954 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 100.62
Current children cumulated vsize (KiB) 934816

[startup+51.1011 s]
/proc/loadavg: 8.00 8.04 8.00 9/186 30818
/proc/meminfo: memFree=12939184/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=1396424 CPUtime=202.54 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 301893 0 0 0 20115 139 0 0 25 0 5 0 517056551 1429938176 300689 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 349106 300691 176 36 0 347011 0
[pid=30811/tid=30813] ppid=30809 vsize=1396424 CPUtime=50.53 cores=0,2,4,6
/proc/30811/task/30813/stat : 30813 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 104093 0 0 0 5030 23 0 0 25 0 5 0 517056578 1429938176 300691 33554432000 4194304 4338100 140736759754352 18446744073709551615 274473664053 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30811/tid=30814] ppid=30809 vsize=1396424 CPUtime=50.71 cores=0,2,4,6
/proc/30811/task/30814/stat : 30814 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 44056 0 0 0 5002 69 0 0 25 0 5 0 517056578 1429938176 300691 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=1396424 CPUtime=50.82 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 96415 0 0 0 5065 17 0 0 25 0 5 0 517056578 1429938176 300691 33554432000 4194304 4338100 140736759754352 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30811/tid=30816] ppid=30809 vsize=1396424 CPUtime=50.18 cores=0,2,4,6
/proc/30811/task/30816/stat : 30816 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 45384 0 0 0 4991 27 0 0 18 0 5 0 517056578 1429938176 300691 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 202.54
Current children cumulated vsize (KiB) 1396424

[startup+102.306 s]
/proc/loadavg: 8.08 8.05 8.00 9/186 30819
/proc/meminfo: memFree=11935404/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=1800484 CPUtime=406.41 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 402752 0 0 0 40400 241 0 0 25 0 5 0 517056551 1843695616 401548 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 450121 401548 176 36 0 448026 0
[pid=30811/tid=30813] ppid=30809 vsize=1800484 CPUtime=101.43 cores=0,2,4,6
/proc/30811/task/30813/stat : 30813 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 147655 0 0 0 10108 35 0 0 25 0 5 0 517056578 1843695616 401548 33554432000 4194304 4338100 140736759754352 18446744073709551615 4206915 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30811/tid=30814] ppid=30809 vsize=1800484 CPUtime=101.88 cores=0,2,4,6
/proc/30811/task/30814/stat : 30814 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 55632 0 0 0 10049 139 0 0 25 0 5 0 517056578 1843695616 401548 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=1800484 CPUtime=102.02 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 131272 0 0 0 10177 25 0 0 25 0 5 0 517056578 1843695616 401548 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30811/tid=30816] ppid=30809 vsize=1800484 CPUtime=100.78 cores=0,2,4,6
/proc/30811/task/30816/stat : 30816 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 56246 0 0 0 10039 39 0 0 18 0 5 0 517056578 1843695616 401548 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 406.41
Current children cumulated vsize (KiB) 1800484

[startup+162.3 s]
/proc/loadavg: 8.09 8.05 8.00 9/186 30821
/proc/meminfo: memFree=10829316/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=2299024 CPUtime=645.36 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 526692 0 0 0 64189 347 0 0 25 0 5 0 517056551 2354200576 525488 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 574756 525488 176 36 0 572661 0
[pid=30811/tid=30813] ppid=30809 vsize=2299024 CPUtime=161.08 cores=0,2,4,6
/proc/30811/task/30813/stat : 30813 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 206158 0 0 0 16058 50 0 0 25 0 5 0 517056578 2354200576 525488 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30811/tid=30814] ppid=30809 vsize=2299024 CPUtime=161.84 cores=0,2,4,6
/proc/30811/task/30814/stat : 30814 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 59800 0 0 0 15973 211 0 0 25 0 5 0 517056578 2354200576 525488 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=2299024 CPUtime=162 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 181452 0 0 0 16167 33 0 0 25 0 5 0 517056578 2354200576 525488 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254285 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30811/tid=30816] ppid=30809 vsize=2299024 CPUtime=160.13 cores=0,2,4,6
/proc/30811/task/30816/stat : 30816 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 67335 0 0 0 15963 50 0 0 18 0 5 0 517056578 2354200576 525488 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 645.36
Current children cumulated vsize (KiB) 2299024


################
# More data... #
################

/proc/30811/task/30816/stat : 30816 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 165678 0 0 0 69359 146 0 0 18 0 5 0 517056578 5412048896 1236491 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2796.59
Current children cumulated vsize (KiB) 5285204

[startup+762.307 s]
/proc/loadavg: 8.03 8.06 8.01 9/186 30837
/proc/meminfo: memFree=4888888/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=5304368 CPUtime=3035.6 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 1242568 0 0 0 302316 1244 0 0 25 0 5 0 517056551 5431672832 1241364 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 1326092 1241364 176 36 0 1323997 0
[pid=30811/tid=30813] ppid=30809 vsize=5304368 CPUtime=757.45 cores=0,2,4,6
/proc/30811/task/30813/stat : 30813 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 476651 0 0 0 75603 142 0 0 25 0 5 0 517056578 5431672832 1241364 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30811/tid=30814] ppid=30809 vsize=5304368 CPUtime=761.54 cores=0,2,4,6
/proc/30811/task/30814/stat : 30814 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 152806 0 0 0 75296 858 0 0 25 0 5 0 517056578 5431672832 1241364 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=5304368 CPUtime=761.75 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 435372 0 0 0 76086 89 0 0 25 0 5 0 517056578 5431672832 1241364 33554432000 4194304 4338100 140736759754352 18446744073709551615 274473363962 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30811/tid=30816] ppid=30809 vsize=5304368 CPUtime=754.55 cores=0,2,4,6
/proc/30811/task/30816/stat : 30816 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 165792 0 0 0 75304 151 0 0 18 0 5 0 517056578 5431672832 1241364 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3035.6
Current children cumulated vsize (KiB) 5304368

[startup+822.3 s]
/proc/loadavg: 8.01 8.05 8.00 9/186 30838
/proc/meminfo: memFree=4428840/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=5520992 CPUtime=3274.55 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 1297490 0 0 0 326112 1343 0 0 25 0 5 0 517056551 5653495808 1296286 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 1380248 1296286 176 36 0 1378153 0
[pid=30811/tid=30813] ppid=30809 vsize=5520992 CPUtime=817.08 cores=0,2,4,6
/proc/30811/task/30813/stat : 30813 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 524154 0 0 0 81549 159 0 0 25 0 5 0 517056578 5653495808 1296286 33554432000 4194304 4338100 140736759754352 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30811/tid=30814] ppid=30809 vsize=5520992 CPUtime=821.48 cores=0,2,4,6
/proc/30811/task/30814/stat : 30814 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 152989 0 0 0 81220 928 0 0 25 0 5 0 517056578 5653495808 1296286 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=5520992 CPUtime=821.74 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 442420 0 0 0 82083 91 0 0 25 0 5 0 517056578 5653495808 1296286 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254321 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30811/tid=30816] ppid=30809 vsize=5520992 CPUtime=813.94 cores=0,2,4,6
/proc/30811/task/30816/stat : 30816 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 165980 0 0 0 81233 161 0 0 18 0 5 0 517056578 5653495808 1296286 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3274.55
Current children cumulated vsize (KiB) 5520992

[startup+882.301 s]
/proc/loadavg: 8.07 8.06 8.00 9/186 30840
/proc/meminfo: memFree=4168192/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=5562448 CPUtime=3513.63 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 1308186 0 0 0 349944 1419 0 0 25 0 5 0 517056551 5695946752 1306982 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 1390612 1306982 176 36 0 1388517 0
[pid=30811/tid=30813] ppid=30809 vsize=5562448 CPUtime=876.73 cores=0,2,4,6
/proc/30811/task/30813/stat : 30813 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 529593 0 0 0 87509 164 0 0 25 0 5 0 517056578 5695946752 1306982 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30811/tid=30814] ppid=30809 vsize=5562448 CPUtime=881.43 cores=0,2,4,6
/proc/30811/task/30814/stat : 30814 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 153163 0 0 0 87154 989 0 0 25 0 5 0 517056578 5695946752 1306982 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=5562448 CPUtime=881.69 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 447336 0 0 0 88077 92 0 0 25 0 5 0 517056578 5695946752 1306982 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254352 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30811/tid=30816] ppid=30809 vsize=5562448 CPUtime=873.46 cores=0,2,4,6
/proc/30811/task/30816/stat : 30816 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 166147 0 0 0 87176 170 0 0 18 0 5 0 517056578 5695946752 1306982 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3513.63
Current children cumulated vsize (KiB) 5562448

[startup+942.3 s]
/proc/loadavg: 8.09 8.06 8.00 9/186 30841
/proc/meminfo: memFree=3792332/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=5763168 CPUtime=3752.75 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 1359022 0 0 0 373778 1497 0 0 25 0 5 0 517056551 5901484032 1357818 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 1440792 1357818 176 36 0 1438697 0
[pid=30811/tid=30813] ppid=30809 vsize=5763168 CPUtime=936.36 cores=0,2,4,6
/proc/30811/task/30813/stat : 30813 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 558523 0 0 0 93461 175 0 0 25 0 5 0 517056578 5901484032 1357818 33554432000 4194304 4338100 140736759754352 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30811/tid=30814] ppid=30809 vsize=5763168 CPUtime=941.41 cores=0,2,4,6
/proc/30811/task/30814/stat : 30814 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 154731 0 0 0 93094 1047 0 0 25 0 5 0 517056578 5901484032 1357818 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=5763168 CPUtime=941.65 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 467524 0 0 0 94068 97 0 0 25 0 5 0 517056578 5901484032 1357818 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30811/tid=30816] ppid=30809 vsize=5763168 CPUtime=933.02 cores=0,2,4,6
/proc/30811/task/30816/stat : 30816 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 166297 0 0 0 93127 175 0 0 18 0 5 0 517056578 5901484032 1357818 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3752.75
Current children cumulated vsize (KiB) 5763168

[startup+1002.3 s]
/proc/loadavg: 8.17 8.08 8.01 9/186 30843
/proc/meminfo: memFree=3307740/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=6139188 CPUtime=3991.81 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 1452905 0 0 0 397604 1577 0 0 25 0 5 0 517056551 6286528512 1451701 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 1534797 1451701 176 36 0 1532702 0
[pid=30811/tid=30813] ppid=30809 vsize=6139188 CPUtime=996.19 cores=0,2,4,6
/proc/30811/task/30813/stat : 30813 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 592125 0 0 0 99434 185 0 0 25 0 5 0 517056578 6286528512 1451701 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254321 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30811/tid=30814] ppid=30809 vsize=6139188 CPUtime=1001 cores=0,2,4,6
/proc/30811/task/30814/stat : 30814 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 171719 0 0 0 99032 1068 0 0 18 0 5 0 517056578 6286528512 1451701 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=6139188 CPUtime=1001.4 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 501343 0 0 0 100035 105 0 0 25 0 5 0 517056578 6286528512 1451701 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30811/tid=30816] ppid=30809 vsize=6139188 CPUtime=992.91 cores=0,2,4,6
/proc/30811/task/30816/stat : 30816 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 175771 0 0 0 99076 215 0 0 25 0 5 0 517056578 6286528512 1451701 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3991.81
Current children cumulated vsize (KiB) 6139188

[startup+1062.3 s]
/proc/loadavg: 8.19 8.09 8.01 9/186 30845
/proc/meminfo: memFree=2961148/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=6230296 CPUtime=4230.75 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 1476830 0 0 0 421433 1642 0 0 25 0 5 0 517056551 6379823104 1475626 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 1557574 1475626 176 36 0 1555479 0
[pid=30811/tid=30813] ppid=30809 vsize=6230296 CPUtime=1056.13 cores=0,2,4,6
/proc/30811/task/30813/stat : 30813 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 603530 0 0 0 105423 190 0 0 25 0 5 0 517056578 6379823104 1475626 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30811/tid=30814] ppid=30809 vsize=6230296 CPUtime=1060.44 cores=0,2,4,6
/proc/30811/task/30814/stat : 30814 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 171837 0 0 0 104971 1073 0 0 18 0 5 0 517056578 6379823104 1475626 33554432000 4194304 4338100 140736759754352 18446744073709551615 4253954 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=6230296 CPUtime=1061.02 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 512029 0 0 0 105994 108 0 0 25 0 5 0 517056578 6379823104 1475626 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254321 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30811/tid=30816] ppid=30809 vsize=6230296 CPUtime=1052.86 cores=0,2,4,6
/proc/30811/task/30816/stat : 30816 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 177487 0 0 0 105018 268 0 0 25 0 5 0 517056578 6379823104 1475626 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4230.75
Current children cumulated vsize (KiB) 6230296

[startup+1122.3 s]
/proc/loadavg: 8.18 8.10 8.02 9/186 30846
/proc/meminfo: memFree=2648772/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=6399868 CPUtime=4469.82 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 1518867 0 0 0 445273 1709 0 0 25 0 5 0 517056551 6553464832 1517663 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 1599967 1517663 176 36 0 1597872 0
[pid=30811/tid=30813] ppid=30809 vsize=6399868 CPUtime=1116.07 cores=0,2,4,6
/proc/30811/task/30813/stat : 30813 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 618904 0 0 0 111410 197 0 0 25 0 5 0 517056578 6553464832 1517663 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30811/tid=30814] ppid=30809 vsize=6399868 CPUtime=1119.97 cores=0,2,4,6
/proc/30811/task/30814/stat : 30814 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 175092 0 0 0 110918 1079 0 0 18 0 5 0 517056578 6553464832 1517663 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=6399868 CPUtime=1120.64 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 527381 0 0 0 111952 112 0 0 25 0 5 0 517056578 6553464832 1517663 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30811/tid=30816] ppid=30809 vsize=6399868 CPUtime=1112.83 cores=0,2,4,6
/proc/30811/task/30816/stat : 30816 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 185543 0 0 0 110965 318 0 0 25 0 5 0 517056578 6553464832 1517663 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4469.82
Current children cumulated vsize (KiB) 6399868

[startup+1182.3 s]
/proc/loadavg: 8.17 8.11 8.02 9/186 30848
/proc/meminfo: memFree=2513604/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=6412908 CPUtime=4709 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) S 30809 30811 30368 0 -1 4202496 1522265 0 0 0 469133 1767 0 0 25 0 5 0 517056551 6566817792 1521061 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 1603227 1521061 176 36 0 1601132 0
[pid=30811/tid=30813] ppid=30809 vsize=6412908 CPUtime=1176.03 cores=0,2,4,6
/proc/30811/task/30813/stat : 30813 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 621957 0 0 0 117402 201 0 0 25 0 5 0 517056578 6566817792 1521061 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254329 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30811/tid=30814] ppid=30809 vsize=6412908 CPUtime=1179.56 cores=0,2,4,6
/proc/30811/task/30814/stat : 30814 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 175223 0 0 0 116871 1085 0 0 18 0 5 0 517056578 6566817792 1521061 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=6412908 CPUtime=1180.29 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 527455 0 0 0 117917 112 0 0 25 0 5 0 517056578 6566817792 1521061 33554432000 4194304 4338100 140736759754352 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30811/tid=30816] ppid=30809 vsize=6412908 CPUtime=1172.8 cores=0,2,4,6
/proc/30811/task/30816/stat : 30816 (ttsth-5-4) R 30809 30811 30368 0 -1 4202560 185683 0 0 0 116915 365 0 0 25 0 5 0 517056578 6566817792 1521061 33554432000 4194304 4338100 140736759754352 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4709
Current children cumulated vsize (KiB) 6412908



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1205.2 s]
/proc/loadavg: 7.79 8.04 8.00 2/174 30867
/proc/meminfo: memFree=10153396/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=0 CPUtime=4800.13 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) Z 30809 30811 30368 0 -1 4203548 1542679 0 0 0 478212 1801 0 0 18 0 2 0 517056551 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 0 0 0 0 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=0 CPUtime=1203.07 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4203588 536080 0 0 0 120187 120 0 0 25 0 2 0 517056578 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.13
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1205.31 s]
/proc/loadavg: 7.79 8.04 8.00 2/174 30867
/proc/meminfo: memFree=10153396/32950928 swapFree=67111528/67111528
[pid=30811] ppid=30809 vsize=0 CPUtime=4800.24 cores=0,2,4,6
/proc/30811/stat : 30811 (ttsth-5-4) Z 30809 30811 30368 0 -1 4203548 1542679 0 0 0 478212 1812 0 0 18 0 2 0 517056551 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/30811/statm: 0 0 0 0 0 0 0
[pid=30811/tid=30815] ppid=30809 vsize=0 CPUtime=1203.17 cores=0,2,4,6
/proc/30811/task/30815/stat : 30815 (ttsth-5-4) R 30809 30811 30368 0 -1 4203588 536080 0 0 0 120187 130 0 0 25 0 2 0 517056578 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1205.38
CPU time (s): 4800.31
CPU user time (s): 4782.12
CPU system time (s): 18.1862
CPU usage (%): 398.241
Max. virtual memory (cumulated for all children) (KiB): 6494500

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4782.12
system time used= 18.1862
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1542679
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 16513
involuntary context switches= 20854

runsolver used 2.01269 second user time and 4.52931 second system time

The end

Launcher Data 

Begin job on node125 at 2011-04-29 07:22:49
IDJOB=3259274
IDBENCH=71540
IDSOLVER=1654
FILE ID=node125/3259274-1304054569
RUNJOBID= node125-1304053363-30688
PBS_JOBID= 13170412
Free space on /tmp= 72764 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/forced/rbsat-v945c61409gyes7.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3259274-1304054569/watcher-3259274-1304054569 -o /tmp/evaluation-result-3259274-1304054569/solver-3259274-1304054569 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259274-1304054569.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 6200c727c8a5a324974090a1f9a6d7ab
RANDOM SEED=1598484405

node125.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5266.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      15096236 kB
Buffers:       2218284 kB
Cached:       14972336 kB
SwapCached:          0 kB
Active:        9110004 kB
Inactive:      8213768 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      15096236 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            1212 kB
Writeback:           0 kB
AnonPages:      132316 kB
Mapped:          15076 kB
Slab:           468456 kB
PageTables:       4244 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   322984 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72764 MiB
End job on node125 at 2011-04-29 07:42:56