Trace number 3262222

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.36 1202.23

General information on the benchmark

Namecrafted/kullmann/VanDerWaerden/VanderWaerden_pd_3k/
VanDerWaerden_pd_2-3-21_401.cnf
MD5SUM4e87573be2b7b7324c13131d6c6896b8
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark194.97
Satisfiable
(Un)Satisfiability was proved
Number of variables201
Number of clauses21509
Sum of the clauses size94265
Maximum clause length21
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2266
Number of clauses of size 319560
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 51683

Solver Data 

0.01/0.02	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262222-1304091723.cnf 
0.01/0.02	c Using SATTIMEOUT of 4800
0.01/0.02	c Using SATRAM of 15500
0.01/0.02	c Size of subset node = 32
0.01/0.02	c Variables = 201
0.01/0.02	c Clauses = 21509
0.01/0.02	c Literals = 94265
0.01/0.02	c Reduced size:
0.01/0.02	c Variables = 201
0.01/0.02	c Clauses = 21509
0.01/0.02	c Literals = 94265
0.01/0.02	c a2
0.09/0.12	c a1
0.19/0.29	c Independent sub-problems = 1
0.19/0.29	c Sub-problem with 201 variables and 21509 clauses
0.19/0.29	c using permutation method 4
0.19/0.29	c count = 1
0.19/0.29	c Ordering finished 0.280 seconds
0.19/0.29	c mapped 0.280 seconds
0.19/0.29	c permuted 0.280 seconds
0.19/0.29	c about to sort
0.19/0.29	c sorted
0.29/0.36	c clauses now 21509
0.29/0.36	c init finished 0.290 seconds
0.29/0.36	c tree built 0.350 seconds
0.29/0.36	c 
0.29/0.36	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.29/0.36	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.29/0.36	c ................................ | ..........                    |                                  0.3         0.0             
0.29/0.36	c ................................ | ..........                  1 |                                  0.3         2.9             
0.29/0.36	c ................................ | ..........                  2 |                                  0.3         5.7             
0.29/0.36	c ................................ | ..........                  3 |                                  0.3         8.6             
0.29/0.36	c ................................ | ..........                  4 |                                  0.3        11.4             
0.29/0.36	c ................................ | ..........                  5 |                                  0.3        14.3             
3.32/1.17	c ................................ | ..........               1000 | 6            360                 0.4      2500.0        17548
95.97/24.33	c ................................ | ..........             469000 | 42410        186716             96.1      4880.3     23194100
192.27/48.43	c ................................ | ..........             716000 | 73873        276726            192.4      3722.2     36535375
288.12/72.42	c ................................ | ..........             916000 | 95629        345952            288.2      3178.3     47190254
383.97/96.47	c ................................ | ..........            1078000 | 112397       405058            384.2      2805.5     51049483
480.24/120.54	c ................................ | *.........            1211000 | 129857       460528            480.4      2520.8     59589965
576.11/144.52	c ................................ | *.........            1335000 | 147847       502539            576.2      2316.8     68448558
671.98/168.53	c ................................ | *.........            1455000 | 166796       541533            672.1      2164.8     77234834
768.65/192.70	c ................................ | *.........            1564000 | 182504       582715            768.7      2034.7     84365856
864.14/216.70	c ................................ | *.........            1672000 | 197874       618247            864.5      1934.0     90891671
960.40/240.73	c ................................ | **........            1774000 | 208281       658614            960.5      1846.9     97354202
1056.26/264.75	c ................................ | **........            1872000 | 220141       691167           1056.5      1771.9    104278990
1152.54/288.89	c ................................ | **........            1970000 | 230849       723092           1152.9      1708.7    109912787
1248.41/312.83	c ................................ | **........            2066000 | 241642       758324           1248.5      1654.7    116024941
1344.28/336.81	c ................................ | **........            2154000 | 251410       789652           1344.3      1602.3    122261399
1440.55/360.95	c ................................ | ***.......            2245000 | 259427       817798           1440.7      1558.2    128013447
1536.44/384.98	c ................................ | ***.......            2337000 | 267127       847838           1536.8      1520.7    132923244
1633.12/409.12	c ................................ | ***.......            2415000 | 274989       875512           1633.2      1478.7    137486474
1728.96/433.11	c ................................ | ***.......            2482000 | 282635       897044           1729.0      1435.5    143405978
1824.42/457.09	c ................................ | ***.......            2550000 | 291675       915924           1824.8      1397.4    148791896
1921.10/481.27	c ................................ | ****......            2615000 | 299904       939297           1921.4      1361.0    154060617
2016.96/505.22	c ................................ | ****......            2671000 | 307550       965164           2017.0      1324.2    159247885
2113.64/529.41	c ................................ | ****......            2728000 | 314899       986744           2113.7      1290.6    163768527
2207.91/553.08	c ................................ | ****......            2785000 | 322529       1007068          2208.2      1261.2    167836376
2305.39/577.50	c ................................ | ****......            2843000 | 330130       1026264          2305.8      1233.0    171883041
2401.25/601.48	c ................................ | *****.....            2891000 | 336942       1043288          2401.6      1203.8    175255777
2497.52/625.50	c ................................ | *****.....            2944000 | 343493       1062255          2497.5      1178.8    178875310
2591.79/649.18	c ................................ | *****.....            2994000 | 350647       1082042          2592.1      1155.0    182845247
2688.09/673.28	c ................................ | *****.....            3044000 | 357550       1101900          2688.4      1132.3    186841984
2784.35/697.37	c ................................ | *****.....            3095000 | 365175       1120654          2784.6      1111.5    190301183
2881.80/721.72	c ................................ | ******....            3147000 | 372769       1139502          2881.9      1092.0    193324175
2977.27/745.64	c ................................ | ******....            3200000 | 379624       1157169          2977.5      1074.7    196723128
3073.15/769.68	c ................................ | ******....            3252000 | 386497       1175539          3073.5      1058.1    200533357
3169.82/793.81	c ................................ | ******....            3302000 | 394286       1193172          3169.9      1041.7    203991421
3263.71/817.38	c ................................ | ******....            3358000 | 400638       1209499          3264.0      1028.8    207437796
3360.76/841.63	c ................................ | *******...            3423000 | 407520       1227942          3360.9      1018.5    211336669
3456.23/865.50	c ................................ | *******...            3486000 | 414908       1245828          3456.2      1008.6    214881300
3553.32/889.83	c ................................ | *******...            3552000 | 422597       1264001          3553.4       999.6    218269526
3648.37/913.63	c ................................ | *******...            3604000 | 428099       1283589          3648.5       987.8    221565686
3745.85/938.04	c ................................ | *******...            3653000 | 434206       1299379          3746.0       975.2    225184469
3840.51/961.72	c ................................ | ********..            3707000 | 440389       1317061          3840.6       965.2    229030921
3936.38/985.79	c ................................ | ********..            3771000 | 445970       1333159          3936.8       957.9    232806141
4032.68/1009.89	c ................................ | ********..            3835000 | 451922       1351586          4033.0       950.9    236032786
4128.53/1033.87	c ................................ | ********..            3899000 | 458370       1369611          4128.8       944.3    239463304
4224.00/1057.71	c ................................ | ********..            3957000 | 464994       1388606          4224.1       936.8    242614624
4319.87/1081.75	c ................................ | *********.            4003000 | 471800       1404311          4320.1       926.6    246052270
4418.14/1106.32	c ................................ | *********.            4047000 | 478761       1421020          4418.2       916.0    249491300
4513.59/1130.29	c ................................ | *********.            4090000 | 484717       1437680          4514.0       906.1    252916487
4609.45/1154.27	c ................................ | *********.            4132000 | 491769       1453874          4609.7       896.4    256423794
4704.54/1178.00	c ................................ | *********.            4177000 | 499503       1466916          4704.5       887.9    260119443

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3262222-1304091723/watcher-3262222-1304091723 -o /tmp/evaluation-result-3262222-1304091723/solver-3262222-1304091723 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262222-1304091723.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.15 8.10 8.02 2/174 7011
/proc/meminfo: memFree=26117084/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=9408 CPUtime=0 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) R 7009 7011 6568 0 -1 4202496 434 0 0 0 0 0 0 0 22 0 1 0 520772746 9633792 376 33554432000 4194304 4338100 140736186756224 18446744073709551615 230808826565 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/7011/statm: 2352 376 146 36 0 257 0

[startup+0.019285 s]
/proc/loadavg: 8.15 8.10 8.02 2/174 7011
/proc/meminfo: memFree=26117084/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=11292 CPUtime=0.01 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) R 7009 7011 6568 0 -1 4202496 888 0 0 0 1 0 0 0 22 0 1 0 520772746 11563008 830 33554432000 4194304 4338100 140736186756224 18446744073709551615 230808826565 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/7011/statm: 2823 830 148 36 0 728 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 11292

[startup+0.100271 s]
/proc/loadavg: 8.15 8.10 8.02 2/174 7011
/proc/meminfo: memFree=26117084/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=23812 CPUtime=0.09 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) R 7009 7011 6568 0 -1 4202496 3410 0 0 0 9 0 0 0 22 0 1 0 520772746 24383488 3252 33554432000 4194304 4338100 140736186756224 18446744073709551615 4206987 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/7011/statm: 5953 3252 325 36 0 3858 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 23812

[startup+0.300232 s]
/proc/loadavg: 8.15 8.10 8.02 2/174 7011
/proc/meminfo: memFree=26117084/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=15136 CPUtime=0.29 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) R 7009 7011 6568 0 -1 4202496 4057 0 0 0 29 0 0 0 23 0 1 0 520772746 15499264 1775 33554432000 4194304 4338100 140736186756224 18446744073709551615 230808826565 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/7011/statm: 3784 1789 163 36 0 1689 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 15136

[startup+0.700129 s]
/proc/loadavg: 8.15 8.10 8.02 2/174 7011
/proc/meminfo: memFree=26117084/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=162572 CPUtime=1.7 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 13046 0 0 0 168 2 0 0 24 0 5 0 520772746 166473728 10721 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 40643 10721 176 36 0 38548 0
Current children cumulated CPU time (s) 1.7
Current children cumulated vsize (KiB) 162572

[startup+1.50098 s]
/proc/loadavg: 8.15 8.10 8.02 10/186 7038
/proc/meminfo: memFree=26009200/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=210284 CPUtime=4.89 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 25078 0 0 0 485 4 0 0 24 0 5 0 520772746 215330816 22753 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 52571 22755 176 36 0 50476 0
[pid=7011/tid=7031] ppid=7009 vsize=210284 CPUtime=1.13 cores=1,3,5,7
/proc/7011/task/7031/stat : 7031 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 4934 0 0 0 113 0 0 0 19 0 5 0 520772782 215330816 22755 33554432000 4194304 4338100 140736186756224 18446744073709551615 4285386 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=210284 CPUtime=1.13 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 3965 0 0 0 112 1 0 0 19 0 5 0 520772782 215330816 22756 33554432000 4194304 4338100 140736186756224 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=7011/tid=7033] ppid=7009 vsize=210284 CPUtime=1.12 cores=1,3,5,7
/proc/7011/task/7033/stat : 7033 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 4593 0 0 0 112 0 0 0 25 0 5 0 520772782 215330816 22756 33554432000 4194304 4338100 140736186756224 18446744073709551615 230808560656 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=7011/tid=7034] ppid=7009 vsize=210284 CPUtime=1.13 cores=1,3,5,7
/proc/7011/task/7034/stat : 7034 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 3950 0 0 0 112 1 0 0 19 0 5 0 520772782 215330816 22757 33554432000 4194304 4338100 140736186756224 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4.89
Current children cumulated vsize (KiB) 210284

[startup+3.10069 s]
/proc/loadavg: 8.15 8.10 8.02 9/186 7038
/proc/meminfo: memFree=25891028/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=282888 CPUtime=11.29 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 43355 0 0 0 1121 8 0 0 24 0 5 0 520772746 289677312 41030 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 70722 41030 176 36 0 68627 0
[pid=7011/tid=7031] ppid=7009 vsize=282888 CPUtime=2.73 cores=1,3,5,7
/proc/7011/task/7031/stat : 7031 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 10932 0 0 0 271 2 0 0 23 0 5 0 520772782 289677312 41030 33554432000 4194304 4338100 140736186756224 18446744073709551615 4287681 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=282888 CPUtime=2.72 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 8766 0 0 0 271 1 0 0 24 0 5 0 520772782 289677312 41030 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=7011/tid=7033] ppid=7009 vsize=282888 CPUtime=2.72 cores=1,3,5,7
/proc/7011/task/7033/stat : 7033 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 8542 0 0 0 271 1 0 0 25 0 5 0 520772782 289677312 41030 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820979971 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=7011/tid=7034] ppid=7009 vsize=282888 CPUtime=2.73 cores=1,3,5,7
/proc/7011/task/7034/stat : 7034 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 7476 0 0 0 271 2 0 0 23 0 5 0 520772782 289677312 41031 33554432000 4194304 4338100 140736186756224 18446744073709551615 4251295 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 11.29
Current children cumulated vsize (KiB) 282888

[startup+6.3011 s]
/proc/loadavg: 8.14 8.09 8.02 9/186 7038
/proc/meminfo: memFree=25609516/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=418316 CPUtime=24.07 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 77330 0 0 0 2393 14 0 0 24 0 5 0 520772746 428355584 75005 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 104579 75006 176 36 0 102484 0
[pid=7011/tid=7031] ppid=7009 vsize=418316 CPUtime=5.93 cores=1,3,5,7
/proc/7011/task/7031/stat : 7031 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 21677 0 0 0 589 4 0 0 25 0 5 0 520772782 428355584 75007 33554432000 4194304 4338100 140736186756224 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=418316 CPUtime=5.92 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 18170 0 0 0 589 3 0 0 25 0 5 0 520772782 428355584 75007 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=7011/tid=7033] ppid=7009 vsize=418316 CPUtime=5.9 cores=1,3,5,7
/proc/7011/task/7033/stat : 7033 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 14691 0 0 0 589 1 0 0 25 0 5 0 520772782 428355584 75007 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=7011/tid=7034] ppid=7009 vsize=418316 CPUtime=5.93 cores=1,3,5,7
/proc/7011/task/7034/stat : 7034 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 15155 0 0 0 590 3 0 0 25 0 5 0 520772782 428355584 75007 33554432000 4194304 4338100 140736186756224 18446744073709551615 4251301 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 24.07
Current children cumulated vsize (KiB) 418316

[startup+12.7009 s]
/proc/loadavg: 8.13 8.09 8.02 9/186 7038
/proc/meminfo: memFree=25225636/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=603924 CPUtime=49.64 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 123283 0 0 0 4941 23 0 0 24 0 5 0 520772746 618418176 120958 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 150981 120958 176 36 0 148886 0
[pid=7011/tid=7031] ppid=7009 vsize=603924 CPUtime=12.34 cores=1,3,5,7
/proc/7011/task/7031/stat : 7031 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 36050 0 0 0 1226 8 0 0 25 0 5 0 520772782 618418176 120958 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=603924 CPUtime=12.32 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 31396 0 0 0 1227 5 0 0 25 0 5 0 520772782 618418176 120958 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=7011/tid=7033] ppid=7009 vsize=603924 CPUtime=12.26 cores=1,3,5,7
/proc/7011/task/7033/stat : 7033 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 23282 0 0 0 1224 2 0 0 25 0 5 0 520772782 618418176 120958 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=7011/tid=7034] ppid=7009 vsize=603924 CPUtime=12.33 cores=1,3,5,7
/proc/7011/task/7034/stat : 7034 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 24916 0 0 0 1228 5 0 0 25 0 5 0 520772782 618418176 120958 33554432000 4194304 4338100 140736186756224 18446744073709551615 4256510 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 49.64
Current children cumulated vsize (KiB) 603924

[startup+25.5006 s]
/proc/loadavg: 8.10 8.09 8.02 9/186 7039
/proc/meminfo: memFree=24632704/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=899012 CPUtime=100.76 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 197451 0 0 0 10037 39 0 0 24 0 5 0 520772746 920588288 195126 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 224753 195126 176 36 0 222658 0
[pid=7011/tid=7031] ppid=7009 vsize=899012 CPUtime=25.13 cores=1,3,5,7
/proc/7011/task/7031/stat : 7031 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 58392 0 0 0 2499 14 0 0 25 0 5 0 520772782 920588288 195126 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=899012 CPUtime=25.12 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 53887 0 0 0 2503 9 0 0 25 0 5 0 520772782 920588288 195126 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254221 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=7011/tid=7033] ppid=7009 vsize=899012 CPUtime=24.99 cores=1,3,5,7
/proc/7011/task/7033/stat : 7033 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 36791 0 0 0 2493 6 0 0 25 0 5 0 520772782 920588288 195126 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=7011/tid=7034] ppid=7009 vsize=899012 CPUtime=25.13 cores=1,3,5,7
/proc/7011/task/7034/stat : 7034 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 40742 0 0 0 2505 8 0 0 25 0 5 0 520772782 920588288 195126 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 100.76
Current children cumulated vsize (KiB) 899012

[startup+51.1009 s]
/proc/loadavg: 8.13 8.09 8.02 9/186 7039
/proc/meminfo: memFree=23806956/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=1344904 CPUtime=203.03 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 308759 0 0 0 20239 64 0 0 24 0 5 0 520772746 1377181696 306434 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 336226 306434 176 36 0 334131 0
[pid=7011/tid=7031] ppid=7009 vsize=1344904 CPUtime=50.73 cores=1,3,5,7
/proc/7011/task/7031/stat : 7031 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 91395 0 0 0 5053 20 0 0 25 0 5 0 520772782 1377181696 306434 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=1344904 CPUtime=50.73 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 87650 0 0 0 5055 18 0 0 25 0 5 0 520772782 1377181696 306434 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=7011/tid=7033] ppid=7009 vsize=1344904 CPUtime=50.45 cores=1,3,5,7
/proc/7011/task/7033/stat : 7033 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 61657 0 0 0 5034 11 0 0 25 0 5 0 520772782 1377181696 306434 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=7011/tid=7034] ppid=7009 vsize=1344904 CPUtime=50.73 cores=1,3,5,7
/proc/7011/task/7034/stat : 7034 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 60418 0 0 0 5061 12 0 0 25 0 5 0 520772782 1377181696 306434 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 203.03
Current children cumulated vsize (KiB) 1344904

[startup+102.306 s]
/proc/loadavg: 8.06 8.08 8.02 9/186 7041
/proc/meminfo: memFree=22647916/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=1880800 CPUtime=407.56 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 440265 0 0 0 40659 97 0 0 24 0 5 0 520772746 1925939200 437940 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 470200 437940 176 36 0 468105 0
[pid=7011/tid=7031] ppid=7009 vsize=1880800 CPUtime=101.93 cores=1,3,5,7
/proc/7011/task/7031/stat : 7031 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 126702 0 0 0 10164 29 0 0 25 0 5 0 520772782 1925939200 437940 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=1880800 CPUtime=101.71 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 131029 0 0 0 10143 28 0 0 25 0 5 0 520772782 1925939200 437940 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254433 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=7011/tid=7033] ppid=7009 vsize=1880800 CPUtime=101.58 cores=1,3,5,7
/proc/7011/task/7033/stat : 7033 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 88606 0 0 0 10140 18 0 0 25 0 5 0 520772782 1925939200 437940 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=7011/tid=7034] ppid=7009 vsize=1880800 CPUtime=101.94 cores=1,3,5,7
/proc/7011/task/7034/stat : 7034 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 86289 0 0 0 10175 19 0 0 25 0 5 0 520772782 1925939200 437940 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 407.56
Current children cumulated vsize (KiB) 1880800

[startup+162.3 s]
/proc/loadavg: 8.17 8.09 8.02 9/186 7042
/proc/meminfo: memFree=21350720/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=2580192 CPUtime=647.21 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 614526 0 0 0 64584 137 0 0 24 0 5 0 520772746 2642116608 612201 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 645048 612201 176 36 0 642953 0
[pid=7011/tid=7031] ppid=7009 vsize=2580192 CPUtime=161.93 cores=1,3,5,7
/proc/7011/task/7031/stat : 7031 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 173969 0 0 0 16153 40 0 0 25 0 5 0 520772782 2642116608 612201 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=2580192 CPUtime=161.69 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 180867 0 0 0 16129 40 0 0 25 0 5 0 520772782 2642116608 612202 33554432000 4194304 4338100 140736186756224 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=7011/tid=7033] ppid=7009 vsize=2580192 CPUtime=161.55 cores=1,3,5,7
/proc/7011/task/7033/stat : 7033 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 129943 0 0 0 16129 26 0 0 25 0 5 0 520772782 2642116608 612202 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=7011/tid=7034] ppid=7009 vsize=2580192 CPUtime=161.66 cores=1,3,5,7
/proc/7011/task/7034/stat : 7034 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 122109 0 0 0 16137 29 0 0 25 0 5 0 520772782 2642116608 612202 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 647.21
Current children cumulated vsize (KiB) 2580192


################
# More data... #
################

/proc/meminfo: memFree=14369620/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=6474276 CPUtime=3044.01 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 1590252 0 0 0 304030 371 0 0 24 0 5 0 520772746 6629658624 1587927 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 1618569 1587927 176 36 0 1616474 0
[pid=7011/tid=7031] ppid=7009 vsize=6474276 CPUtime=760.3 cores=1,3,5,7
/proc/7011/task/7031/stat : 7031 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 420868 0 0 0 75928 102 0 0 25 0 5 0 520772782 6629658624 1587927 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254439 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=6474276 CPUtime=761.71 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 486821 0 0 0 76061 110 0 0 25 0 5 0 520772782 6629658624 1587927 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=7011/tid=7033] ppid=7009 vsize=6474276 CPUtime=759.93 cores=1,3,5,7
/proc/7011/task/7033/stat : 7033 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 309762 0 0 0 75928 65 0 0 25 0 5 0 520772782 6629658624 1587927 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=7011/tid=7034] ppid=7009 vsize=6474276 CPUtime=761.67 cores=1,3,5,7
/proc/7011/task/7034/stat : 7034 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 365162 0 0 0 76076 91 0 0 25 0 5 0 520772782 6629658624 1587927 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3044.01
Current children cumulated vsize (KiB) 6474276

[startup+822.3 s]
/proc/loadavg: 8.19 8.10 8.02 9/186 7060
/proc/meminfo: memFree=13847800/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=6743944 CPUtime=3283.66 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 1658575 0 0 0 327978 388 0 0 24 0 5 0 520772746 6905798656 1656250 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 1685986 1656250 176 36 0 1683891 0
[pid=7011/tid=7031] ppid=7009 vsize=6743944 CPUtime=819.98 cores=1,3,5,7
/proc/7011/task/7031/stat : 7031 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 442597 0 0 0 81889 109 0 0 25 0 5 0 520772782 6905798656 1656250 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254462 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=6743944 CPUtime=821.71 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 506937 0 0 0 82056 115 0 0 25 0 5 0 520772782 6905798656 1656250 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=7011/tid=7033] ppid=7009 vsize=6743944 CPUtime=819.93 cores=1,3,5,7
/proc/7011/task/7033/stat : 7033 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 323657 0 0 0 81925 68 0 0 25 0 5 0 520772782 6905798656 1656250 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=7011/tid=7034] ppid=7009 vsize=6743944 CPUtime=821.67 cores=1,3,5,7
/proc/7011/task/7034/stat : 7034 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 377745 0 0 0 82073 94 0 0 25 0 5 0 520772782 6905798656 1656250 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3283.66
Current children cumulated vsize (KiB) 6743944

[startup+882.3 s]
/proc/loadavg: 8.11 8.09 8.02 9/186 7061
/proc/meminfo: memFree=13426544/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=7032380 CPUtime=3523.34 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 1730288 0 0 0 351930 404 0 0 24 0 5 0 520772746 7201157120 1727963 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 1758095 1727963 176 36 0 1756000 0
[pid=7011/tid=7031] ppid=7009 vsize=7032380 CPUtime=879.64 cores=1,3,5,7
/proc/7011/task/7031/stat : 7031 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 465942 0 0 0 87851 113 0 0 25 0 5 0 520772782 7201157120 1727963 33554432000 4194304 4338100 140736186756224 18446744073709551615 4256993 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=7032380 CPUtime=881.7 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 525893 0 0 0 88051 119 0 0 25 0 5 0 520772782 7201157120 1727963 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254429 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=7011/tid=7033] ppid=7009 vsize=7032380 CPUtime=879.93 cores=1,3,5,7
/proc/7011/task/7033/stat : 7033 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 341116 0 0 0 87921 72 0 0 25 0 5 0 520772782 7201157120 1727963 33554432000 4194304 4338100 140736186756224 18446744073709551615 4256403 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=7011/tid=7034] ppid=7009 vsize=7032380 CPUtime=881.67 cores=1,3,5,7
/proc/7011/task/7034/stat : 7034 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 389698 0 0 0 88069 98 0 0 25 0 5 0 520772782 7201157120 1727963 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3523.34
Current children cumulated vsize (KiB) 7032380

[startup+942.3 s]
/proc/loadavg: 8.08 8.09 8.02 9/186 7063
/proc/meminfo: memFree=13123204/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=7305176 CPUtime=3763.03 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 1798204 0 0 0 375880 423 0 0 24 0 5 0 520772746 7480500224 1795879 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 1826294 1795879 176 36 0 1824199 0
[pid=7011/tid=7031] ppid=7009 vsize=7305176 CPUtime=939.32 cores=1,3,5,7
/proc/7011/task/7031/stat : 7031 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 487576 0 0 0 93813 119 0 0 25 0 5 0 520772782 7480500224 1795879 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=7305176 CPUtime=941.7 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 543872 0 0 0 94046 124 0 0 25 0 5 0 520772782 7480500224 1795879 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=7011/tid=7033] ppid=7009 vsize=7305176 CPUtime=939.93 cores=1,3,5,7
/proc/7011/task/7033/stat : 7033 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 357069 0 0 0 93918 75 0 0 25 0 5 0 520772782 7480500224 1795879 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254418 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=7011/tid=7034] ppid=7009 vsize=7305176 CPUtime=941.67 cores=1,3,5,7
/proc/7011/task/7034/stat : 7034 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 402048 0 0 0 94065 102 0 0 25 0 5 0 520772782 7480500224 1795879 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3763.03
Current children cumulated vsize (KiB) 7305176

[startup+1002.3 s]
/proc/loadavg: 8.06 8.08 8.02 9/186 7065
/proc/meminfo: memFree=12834872/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=7584096 CPUtime=4002.7 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 1868986 0 0 0 399828 442 0 0 24 0 5 0 520772746 7766114304 1866661 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 1896024 1866661 176 36 0 1893929 0
[pid=7011/tid=7031] ppid=7009 vsize=7584096 CPUtime=999.07 cores=1,3,5,7
/proc/7011/task/7031/stat : 7031 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 508577 0 0 0 99783 124 0 0 25 0 5 0 520772782 7766114304 1866661 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=7584096 CPUtime=1001.63 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 561526 0 0 0 100033 130 0 0 25 0 5 0 520772782 7766114304 1866661 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=7011/tid=7033] ppid=7009 vsize=7584096 CPUtime=999.94 cores=1,3,5,7
/proc/7011/task/7033/stat : 7033 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 371240 0 0 0 99915 79 0 0 25 0 5 0 520772782 7766114304 1866661 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=7011/tid=7034] ppid=7009 vsize=7584096 CPUtime=1001.67 cores=1,3,5,7
/proc/7011/task/7034/stat : 7034 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 420004 0 0 0 100061 106 0 0 25 0 5 0 520772782 7766114304 1866661 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4002.7
Current children cumulated vsize (KiB) 7584096

[startup+1062.3 s]
/proc/loadavg: 8.02 8.06 8.01 9/186 7066
/proc/meminfo: memFree=12574688/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=7847640 CPUtime=4242.38 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 1934094 0 0 0 423778 460 0 0 24 0 5 0 520772746 8035983360 1931769 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 1961910 1931769 176 36 0 1959815 0
[pid=7011/tid=7031] ppid=7009 vsize=7847640 CPUtime=1059.08 cores=1,3,5,7
/proc/7011/task/7031/stat : 7031 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 525415 0 0 0 105778 130 0 0 25 0 5 0 520772782 8035983360 1931769 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=7847640 CPUtime=1061.3 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 578839 0 0 0 105996 134 0 0 25 0 5 0 520772782 8035983360 1931769 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=7011/tid=7033] ppid=7009 vsize=7847640 CPUtime=1059.94 cores=1,3,5,7
/proc/7011/task/7033/stat : 7033 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 387467 0 0 0 105911 83 0 0 25 0 5 0 520772782 8035983360 1931769 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254429 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=7011/tid=7034] ppid=7009 vsize=7847640 CPUtime=1061.67 cores=1,3,5,7
/proc/7011/task/7034/stat : 7034 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 434734 0 0 0 106057 110 0 0 25 0 5 0 520772782 8035983360 1931769 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254337 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4242.38
Current children cumulated vsize (KiB) 7847640

[startup+1122.3 s]
/proc/loadavg: 8.04 8.06 8.01 9/186 7068
/proc/meminfo: memFree=12301244/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=8114048 CPUtime=4482.03 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 2001433 0 0 0 447727 476 0 0 24 0 5 0 520772746 8308785152 1999108 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 2028512 1999108 176 36 0 2026417 0
[pid=7011/tid=7031] ppid=7009 vsize=8114048 CPUtime=1119.08 cores=1,3,5,7
/proc/7011/task/7031/stat : 7031 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 539902 0 0 0 111774 134 0 0 25 0 5 0 520772782 8308785152 1999108 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=8114048 CPUtime=1120.97 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 608851 0 0 0 111957 140 0 0 25 0 5 0 520772782 8308785152 1999108 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=7011/tid=7033] ppid=7009 vsize=8114048 CPUtime=1119.94 cores=1,3,5,7
/proc/7011/task/7033/stat : 7033 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 399885 0 0 0 111908 86 0 0 25 0 5 0 520772782 8308785152 1999108 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=7011/tid=7034] ppid=7009 vsize=8114048 CPUtime=1121.65 cores=1,3,5,7
/proc/7011/task/7034/stat : 7034 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 445156 0 0 0 112052 113 0 0 25 0 5 0 520772782 8308785152 1999108 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4482.03
Current children cumulated vsize (KiB) 8114048

[startup+1182.3 s]
/proc/loadavg: 8.12 8.08 8.02 9/186 7078
/proc/meminfo: memFree=11987152/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=8393100 CPUtime=4721.7 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 2070141 0 0 0 471675 495 0 0 24 0 5 0 520772746 8594534400 2067816 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 2098275 2067816 176 36 0 2096180 0
[pid=7011/tid=7031] ppid=7009 vsize=8393100 CPUtime=1179.07 cores=1,3,5,7
/proc/7011/task/7031/stat : 7031 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 555000 0 0 0 117768 139 0 0 25 0 5 0 520772782 8594534400 2067816 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=8393100 CPUtime=1180.65 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 637980 0 0 0 117917 148 0 0 25 0 5 0 520772782 8594534400 2067816 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=7011/tid=7033] ppid=7009 vsize=8393100 CPUtime=1179.94 cores=1,3,5,7
/proc/7011/task/7033/stat : 7033 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 413551 0 0 0 117905 89 0 0 25 0 5 0 520772782 8594534400 2067816 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=7011/tid=7034] ppid=7009 vsize=8393100 CPUtime=1181.64 cores=1,3,5,7
/proc/7011/task/7034/stat : 7034 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 455971 0 0 0 118048 116 0 0 25 0 5 0 520772782 8594534400 2067816 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4721.7
Current children cumulated vsize (KiB) 8393100



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.9 s]
/proc/loadavg: 8.09 8.07 8.02 9/186 7079
/proc/meminfo: memFree=11865740/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=8477688 CPUtime=4800.01 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) S 7009 7011 6568 0 -1 4202496 2092146 0 0 0 479500 501 0 0 24 0 5 0 520772746 8681152512 2089821 33554432000 4194304 4338100 140736186756224 18446744073709551615 230820968421 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 2119422 2089821 176 36 0 2117327 0
[pid=7011/tid=7031] ppid=7009 vsize=8477688 CPUtime=1198.68 cores=1,3,5,7
/proc/7011/task/7031/stat : 7031 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 560536 0 0 0 119728 140 0 0 25 0 5 0 520772782 8681152512 2089821 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254221 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=8477688 CPUtime=1200.14 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 645453 0 0 0 119864 150 0 0 25 0 5 0 520772782 8681152512 2089821 33554432000 4194304 4338100 140736186756224 18446744073709551615 4229577 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=7011/tid=7033] ppid=7009 vsize=8477688 CPUtime=1199.55 cores=1,3,5,7
/proc/7011/task/7033/stat : 7033 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 418278 0 0 0 119864 91 0 0 25 0 5 0 520772782 8681152512 2089821 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=7011/tid=7034] ppid=7009 vsize=8477688 CPUtime=1201.25 cores=1,3,5,7
/proc/7011/task/7034/stat : 7034 (ttsth-5-4) R 7009 7011 6568 0 -1 4202560 460240 0 0 0 120008 117 0 0 25 0 5 0 520772782 8681152512 2089821 33554432000 4194304 4338100 140736186756224 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.01
Current children cumulated vsize (KiB) 8477688

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202.2 s]
/proc/loadavg: 7.84 8.02 8.00 3/182 7081
/proc/meminfo: memFree=19809064/32951124 swapFree=67111524/67111528
[pid=7011] ppid=7009 vsize=0 CPUtime=4800.33 cores=1,3,5,7
/proc/7011/stat : 7011 (ttsth-5-4) Z 7009 7011 6568 0 -1 4203548 2092146 0 0 0 479503 530 0 0 18 0 2 0 520772746 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7011/statm: 0 0 0 0 0 0 0
[pid=7011/tid=7032] ppid=7009 vsize=0 CPUtime=1200.44 cores=1,3,5,7
/proc/7011/task/7032/stat : 7032 (ttsth-5-4) R 7009 7011 6568 0 -1 4203588 645453 0 0 0 119864 180 0 0 25 0 2 0 520772782 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.33
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.23
CPU time (s): 4800.36
CPU user time (s): 4795.03
CPU system time (s): 5.33019
CPU usage (%): 399.287
Max. virtual memory (cumulated for all children) (KiB): 8477688

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4795.03
system time used= 5.33019
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2092146
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 661
involuntary context switches= 31903

runsolver used 2.04369 second user time and 4.52931 second system time

The end

Launcher Data 

Begin job on node104 at 2011-04-29 17:42:03
IDJOB=3262222
IDBENCH=83285
IDSOLVER=1654
FILE ID=node104/3262222-1304091723
RUNJOBID= node104-1304090518-6914
PBS_JOBID= 13170552
Free space on /tmp= 71548 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/kullmann/VanDerWaerden/VanderWaerden_pd_3k/VanDerWaerden_pd_2-3-21_401.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3262222-1304091723/watcher-3262222-1304091723 -o /tmp/evaluation-result-3262222-1304091723/solver-3262222-1304091723 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262222-1304091723.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 4e87573be2b7b7324c13131d6c6896b8
RANDOM SEED=2121569735

node104.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5293.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      26117364 kB
Buffers:       1882140 kB
Cached:        4344164 kB
SwapCached:          4 kB
Active:         158224 kB
Inactive:      6132568 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      26117364 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:             408 kB
Writeback:           0 kB
AnonPages:       63944 kB
Mapped:          14684 kB
Slab:           479760 kB
PageTables:       4640 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   188560 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71548 MiB
End job on node104 at 2011-04-29 18:02:07