Trace number 3285251

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.5 1207.46

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
transport-transport-city-sequential-35nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.040-NOTKNOWN.cnf
MD5SUMce0d180356398dfbcfd2bbbb1cb3f9e5
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark375.866
Satisfiable
(Un)Satisfiability was proved
Number of variables1260306
Number of clauses6771840
Sum of the clauses size16095280
Maximum clause length982
Minimum clause length1
Number of clauses of size 1720
Number of clauses of size 26714640
Number of clauses of size 3320
Number of clauses of size 41280
Number of clauses of size 53200
Number of clauses of size over 551680

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3285251-1304058621/watcher-3285251-1304058621 -o /tmp/evaluation-result-3285251-1304058621/solver-3285251-1304058621 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3285251-1304058621.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.73 7.65 7.82 5/181 29529
/proc/meminfo: memFree=19138396/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=18576 CPUtime=0 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) R 29527 29529 28898 0 -1 4202496 1899 1251 0 0 0 0 0 0 22 0 1 0 596093171 19021824 1828 33554432000 4194304 4417332 140735103880576 18446744073709551615 251367240797 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/29529/statm: 4644 1840 138 55 0 2530 0

[startup+0.0340761 s]
/proc/loadavg: 6.73 7.65 7.82 5/181 29529
/proc/meminfo: memFree=19138396/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=28124 CPUtime=0.02 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) R 29527 29529 28898 0 -1 4202496 5124 1251 0 0 2 0 0 0 22 0 1 0 596093171 28798976 5053 33554432000 4194304 4417332 140735103880576 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/29529/statm: 7031 5053 147 55 0 4917 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 28124

[startup+0.101071 s]
/proc/loadavg: 6.73 7.65 7.82 5/181 29529
/proc/meminfo: memFree=19138396/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=35832 CPUtime=0.09 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) R 29527 29529 28898 0 -1 4202496 7041 1251 0 0 9 0 0 0 22 0 1 0 596093171 36691968 6965 33554432000 4194304 4417332 140735103880576 18446744073709551615 4209658 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/29529/statm: 8958 6965 148 55 0 6844 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 35832

[startup+0.30103 s]
/proc/loadavg: 6.73 7.65 7.82 5/181 29529
/proc/meminfo: memFree=19138396/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=68896 CPUtime=0.28 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) R 29527 29529 28898 0 -1 4202496 15323 1251 0 0 27 1 0 0 23 0 1 0 596093171 70549504 15247 33554432000 4194304 4417332 140735103880576 18446744073709551615 251367546624 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/29529/statm: 17224 15247 148 55 0 15110 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 68896

[startup+0.700957 s]
/proc/loadavg: 6.73 7.65 7.82 5/181 29529
/proc/meminfo: memFree=19138396/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=118436 CPUtime=0.68 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) R 29527 29529 28898 0 -1 4202496 27745 1251 0 0 65 3 0 0 25 0 1 0 596093171 121278464 27632 33554432000 4194304 4417332 140735103880576 18446744073709551615 4259053 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/29529/statm: 29609 27632 148 55 0 27495 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 118436

[startup+1.50082 s]
/proc/loadavg: 6.59 7.60 7.80 7/182 29535
/proc/meminfo: memFree=18985784/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=257640 CPUtime=1.48 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) R 29527 29529 28898 0 -1 4202496 62691 1251 0 0 140 8 0 0 25 0 1 0 596093171 263823360 62433 33554432000 4194304 4417332 140735103880576 18446744073709551615 251367546624 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/29529/statm: 64410 62433 148 55 0 62296 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 257640

[startup+3.10053 s]
/proc/loadavg: 6.59 7.60 7.80 7/182 29535
/proc/meminfo: memFree=18867364/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=480872 CPUtime=3.09 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) R 29527 29529 28898 0 -1 4202496 118499 1251 0 0 293 16 0 0 25 0 1 0 596093171 492412928 118241 33554432000 4194304 4417332 140735103880576 18446744073709551615 4231537 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/29529/statm: 120218 118241 148 55 0 118104 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 480872

[startup+6.30092 s]
/proc/loadavg: 6.46 7.56 7.79 7/182 29536
/proc/meminfo: memFree=18476068/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=927336 CPUtime=6.29 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) R 29527 29529 28898 0 -1 4202496 230115 1251 0 0 599 30 0 0 25 0 1 0 596093171 949592064 229857 33554432000 4194304 4417332 140735103880576 18446744073709551615 251367546624 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/29529/statm: 231834 229857 148 55 0 229720 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 927336

[startup+12.7007 s]
/proc/loadavg: 6.58 7.57 7.79 10/186 29540
/proc/meminfo: memFree=18034736/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=1186100 CPUtime=28.46 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) S 29527 29529 28898 0 -1 4202496 464110 1251 0 0 2778 68 0 0 25 0 5 0 596093171 1214566400 273101 33554432000 4194304 4417332 140735103880576 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29529/statm: 296525 273127 176 55 0 294411 0
[pid=29529/tid=29537] ppid=29527 vsize=1186100 CPUtime=5.27 cores=1,3,5,7
/proc/29529/task/29537/stat : 29537 (plingeling) R 29527 29529 28898 0 -1 4202560 34666 1251 0 0 521 6 0 0 25 0 5 0 596093913 1214566400 273140 33554432000 4194304 4417332 140735103880576 18446744073709551615 4259140 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29529/tid=29538] ppid=29527 vsize=1186100 CPUtime=5.26 cores=1,3,5,7
/proc/29529/task/29538/stat : 29538 (plingeling) R 29527 29529 28898 0 -1 4202560 34689 1251 0 0 520 6 0 0 25 0 5 0 596093913 1214566400 273155 33554432000 4194304 4417332 140735103880576 18446744073709551615 4302036 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29529/tid=29539] ppid=29527 vsize=1186100 CPUtime=5.26 cores=1,3,5,7
/proc/29529/task/29539/stat : 29539 (plingeling) R 29527 29529 28898 0 -1 4202560 65321 1251 0 0 516 10 0 0 25 0 5 0 596093913 1214566400 273169 33554432000 4194304 4417332 140735103880576 18446744073709551615 251367240797 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29529/tid=29540] ppid=29527 vsize=1186100 CPUtime=5.24 cores=1,3,5,7
/proc/29529/task/29540/stat : 29540 (plingeling) R 29527 29529 28898 0 -1 4202560 25647 1251 0 0 519 5 0 0 25 0 5 0 596093913 1214566400 273185 33554432000 4194304 4417332 140735103880576 18446744073709551615 4320811 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 28.46
Current children cumulated vsize (KiB) 1186100

[startup+25.5003 s]
/proc/loadavg: 6.90 7.59 7.79 10/186 29540
/proc/meminfo: memFree=17889996/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=1245116 CPUtime=79.57 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) S 29527 29529 28898 0 -1 4202496 851971 1251 0 0 7827 130 0 0 25 0 5 0 596093171 1274998784 293204 33554432000 4194304 4417332 140735103880576 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29529/statm: 311279 293204 178 55 0 309165 0
[pid=29529/tid=29537] ppid=29527 vsize=1245116 CPUtime=18.06 cores=1,3,5,7
/proc/29529/task/29537/stat : 29537 (plingeling) R 29527 29529 28898 0 -1 4202560 181951 1251 0 0 1777 29 0 0 25 0 5 0 596093913 1274998784 293204 33554432000 4194304 4417332 140735103880576 18446744073709551615 4331716 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29529/tid=29538] ppid=29527 vsize=1245116 CPUtime=18.05 cores=1,3,5,7
/proc/29529/task/29538/stat : 29538 (plingeling) R 29527 29529 28898 0 -1 4202560 182239 1251 0 0 1776 29 0 0 25 0 5 0 596093913 1274998784 293204 33554432000 4194304 4417332 140735103880576 18446744073709551615 4216240 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29529/tid=29539] ppid=29527 vsize=1245116 CPUtime=18.06 cores=1,3,5,7
/proc/29529/task/29539/stat : 29539 (plingeling) R 29527 29529 28898 0 -1 4202560 157997 1251 0 0 1780 26 0 0 25 0 5 0 596093913 1274998784 293204 33554432000 4194304 4417332 140735103880576 18446744073709551615 4254468 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29529/tid=29540] ppid=29527 vsize=1245116 CPUtime=17.96 cores=1,3,5,7
/proc/29529/task/29540/stat : 29540 (plingeling) R 29527 29529 28898 0 -1 4202560 25929 1251 0 0 1791 5 0 0 25 0 5 0 596093913 1274998784 293204 33554432000 4194304 4417332 140735103880576 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 79.57
Current children cumulated vsize (KiB) 1245116

[startup+51.1006 s]
/proc/loadavg: 7.40 7.65 7.81 10/186 29541
/proc/meminfo: memFree=17656284/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=1387548 CPUtime=181.83 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) S 29527 29529 28898 0 -1 4202496 931679 1251 0 0 18043 140 0 0 25 0 5 0 596093171 1420849152 324731 33554432000 4194304 4417332 140735103880576 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29529/statm: 346887 324731 180 55 0 344773 0
[pid=29529/tid=29537] ppid=29527 vsize=1387548 CPUtime=43.66 cores=1,3,5,7
/proc/29529/task/29537/stat : 29537 (plingeling) R 29527 29529 28898 0 -1 4202560 190342 1251 0 0 4336 30 0 0 25 0 5 0 596093913 1420849152 324731 33554432000 4194304 4417332 140735103880576 18446744073709551615 4258188 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29529/tid=29538] ppid=29527 vsize=1387548 CPUtime=43.65 cores=1,3,5,7
/proc/29529/task/29538/stat : 29538 (plingeling) R 29527 29529 28898 0 -1 4202560 190632 1251 0 0 4335 30 0 0 25 0 5 0 596093913 1420849152 324731 33554432000 4194304 4417332 140735103880576 18446744073709551615 4258589 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29529/tid=29539] ppid=29527 vsize=1387548 CPUtime=43.66 cores=1,3,5,7
/proc/29529/task/29539/stat : 29539 (plingeling) R 29527 29529 28898 0 -1 4202560 176309 1251 0 0 4338 28 0 0 25 0 5 0 596093913 1420849152 324731 33554432000 4194304 4417332 140735103880576 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29529/tid=29540] ppid=29527 vsize=1387548 CPUtime=43.42 cores=1,3,5,7
/proc/29529/task/29540/stat : 29540 (plingeling) R 29527 29529 28898 0 -1 4202560 70541 1251 0 0 4331 11 0 0 25 0 5 0 596093913 1420849152 324731 33554432000 4194304 4417332 140735103880576 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 181.83
Current children cumulated vsize (KiB) 1387548

[startup+102.307 s]
/proc/loadavg: 7.74 7.70 7.82 9/186 29542
/proc/meminfo: memFree=17446908/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=1458132 CPUtime=386.37 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) S 29527 29529 28898 0 -1 4202496 984496 1251 0 0 38489 148 0 0 25 0 5 0 596093171 1493127168 342378 33554432000 4194304 4417332 140735103880576 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29529/statm: 364533 342378 181 55 0 362419 0
[pid=29529/tid=29537] ppid=29527 vsize=1458132 CPUtime=94.88 cores=1,3,5,7
/proc/29529/task/29537/stat : 29537 (plingeling) R 29527 29529 28898 0 -1 4202560 197069 1251 0 0 9456 32 0 0 25 0 5 0 596093913 1493127168 342378 33554432000 4194304 4417332 140735103880576 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29529/tid=29538] ppid=29527 vsize=1458132 CPUtime=94.85 cores=1,3,5,7
/proc/29529/task/29538/stat : 29538 (plingeling) R 29527 29529 28898 0 -1 4202560 194472 1251 0 0 9454 31 0 0 25 0 5 0 596093913 1493127168 342378 33554432000 4194304 4417332 140735103880576 18446744073709551615 4245610 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29529/tid=29539] ppid=29527 vsize=1458132 CPUtime=94.86 cores=1,3,5,7
/proc/29529/task/29539/stat : 29539 (plingeling) R 29527 29529 28898 0 -1 4202560 181303 1251 0 0 9458 28 0 0 25 0 5 0 596093913 1493127168 342378 33554432000 4194304 4417332 140735103880576 18446744073709551615 4237214 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29529/tid=29540] ppid=29527 vsize=1458132 CPUtime=94.35 cores=1,3,5,7
/proc/29529/task/29540/stat : 29540 (plingeling) R 29527 29529 28898 0 -1 4202560 107797 1251 0 0 9419 16 0 0 25 0 5 0 596093913 1493127168 342378 33554432000 4194304 4417332 140735103880576 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 386.37
Current children cumulated vsize (KiB) 1458132

[startup+162.301 s]
/proc/loadavg: 7.90 7.75 7.82 10/186 29544
/proc/meminfo: memFree=17086872/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=1612096 CPUtime=626.02 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) S 29527 29529 28898 0 -1 4202496 1132357 1251 0 0 62433 169 0 0 25 0 5 0 596093171 1650786304 380869 33554432000 4194304 4417332 140735103880576 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29529/statm: 403024 380869 181 55 0 400910 0
[pid=29529/tid=29537] ppid=29527 vsize=1612096 CPUtime=154.87 cores=1,3,5,7
/proc/29529/task/29537/stat : 29537 (plingeling) R 29527 29529 28898 0 -1 4202560 237744 1251 0 0 15450 37 0 0 25 0 5 0 596093913 1650786304 380869 33554432000 4194304 4417332 140735103880576 18446744073709551615 4365963 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29529/tid=29538] ppid=29527 vsize=1612096 CPUtime=154.83 cores=1,3,5,7
/proc/29529/task/29538/stat : 29538 (plingeling) R 29527 29529 28898 0 -1 4202560 244533 1251 0 0 15444 39 0 0 25 0 5 0 596093913 1650786304 380869 33554432000 4194304 4417332 140735103880576 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29529/tid=29539] ppid=29527 vsize=1612096 CPUtime=154.85 cores=1,3,5,7
/proc/29529/task/29539/stat : 29539 (plingeling) R 29527 29529 28898 0 -1 4202560 214737 1251 0 0 15452 33 0 0 25 0 5 0 596093913 1650786304 380869 33554432000 4194304 4417332 140735103880576 18446744073709551615 4232595 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29529/tid=29540] ppid=29527 vsize=1612096 CPUtime=154.01 cores=1,3,5,7
/proc/29529/task/29540/stat : 29540 (plingeling) R 29527 29529 28898 0 -1 4202560 131488 1251 0 0 15383 18 0 0 25 0 5 0 596093913 1650786304 380869 33554432000 4194304 4417332 140735103880576 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 626.02
Current children cumulated vsize (KiB) 1612096

[startup+222.301 s]
/proc/loadavg: 7.96 7.80 7.83 10/186 29545
/proc/meminfo: memFree=16905756/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=1735040 CPUtime=865.69 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) S 29527 29529 28898 0 -1 4202496 1218310 1251 0 0 86388 181 0 0 25 0 5 0 596093171 1776680960 411605 33554432000 4194304 4417332 140735103880576 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29529/statm: 433760 411605 181 55 0 431646 0
[pid=29529/tid=29537] ppid=29527 vsize=1735040 CPUtime=214.87 cores=1,3,5,7
/proc/29529/task/29537/stat : 29537 (plingeling) R 29527 29529 28898 0 -1 4202560 237744 1251 0 0 21450 37 0 0 25 0 5 0 596093913 1776680960 411605 33554432000 4194304 4417332 140735103880576 18446744073709551615 4254468 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29529/tid=29538] ppid=29527 vsize=1735040 CPUtime=214.84 cores=1,3,5,7
/proc/29529/task/29538/stat : 29538 (plingeling) R 29527 29529 28898 0 -1 4202560 253138 1251 0 0 21443 41 0 0 25 0 5 0 596093913 1776680960 411605 33554432000 4194304 4417332 140735103880576 18446744073709551615 4253475 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29529/tid=29539] ppid=29527 vsize=1735040 CPUtime=214.86 cores=1,3,5,7
/proc/29529/task/29539/stat : 29539 (plingeling) R 29527 29529 28898 0 -1 4202560 218828 1251 0 0 21452 34 0 0 25 0 5 0 596093913 1776680960 411605 33554432000 4194304 4417332 140735103880576 18446744073709551615 4364568 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29529/tid=29540] ppid=29527 vsize=1735040 CPUtime=213.69 cores=1,3,5,7
/proc/29529/task/29540/stat : 29540 (plingeling) R 29527 29529 28898 0 -1 4202560 204745 1251 0 0 21341 28 0 0 25 0 5 0 596093913 1776680960 411605 33554432000 4194304 4417332 140735103880576 18446744073709551615 4356771 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 865.69
Current children cumulated vsize (KiB) 1735040

[startup+282.301 s]
/proc/loadavg: 8.05 7.85 7.84 10/186 29547
/proc/meminfo: memFree=16334668/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=1864696 CPUtime=1105.37 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) S 29527 29529 28898 0 -1 4202496 1277898 1251 0 0 110350 187 0 0 25 0 5 0 596093171 1909448704 444019 33554432000 4194304 4417332 140735103880576 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29529/statm: 466174 444019 181 55 0 464060 0
[pid=29529/tid=29537] ppid=29527 vsize=1864696 CPUtime=274.88 cores=1,3,5,7

################
# More data... #
################

/proc/meminfo: memFree=16116052/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=2174632 CPUtime=3022.73 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) S 29527 29529 28898 0 -1 4202496 1868227 1251 0 0 302003 270 0 0 25 0 5 0 596093171 2226823168 521415 33554432000 4194304 4417332 140735103880576 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29529/statm: 543658 521415 181 55 0 541544 0
[pid=29529/tid=29537] ppid=29527 vsize=2174632 CPUtime=754.9 cores=1,3,5,7
/proc/29529/task/29537/stat : 29537 (plingeling) R 29527 29529 28898 0 -1 4202560 294042 1251 0 0 75444 46 0 0 25 0 5 0 596093913 2226823168 521415 33554432000 4194304 4417332 140735103880576 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29529/tid=29538] ppid=29527 vsize=2174632 CPUtime=753.79 cores=1,3,5,7
/proc/29529/task/29538/stat : 29538 (plingeling) R 29527 29529 28898 0 -1 4202560 291507 1251 0 0 75334 45 0 0 25 0 5 0 596093913 2226823168 521415 33554432000 4194304 4417332 140735103880576 18446744073709551615 4232680 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29529/tid=29539] ppid=29527 vsize=2174632 CPUtime=754.85 cores=1,3,5,7
/proc/29529/task/29539/stat : 29539 (plingeling) R 29527 29529 28898 0 -1 4202560 336309 1251 0 0 75435 50 0 0 25 0 5 0 596093913 2226823168 521415 33554432000 4194304 4417332 140735103880576 18446744073709551615 4265937 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29529/tid=29540] ppid=29527 vsize=2174632 CPUtime=751.76 cores=1,3,5,7
/proc/29529/task/29540/stat : 29540 (plingeling) R 29527 29529 28898 0 -1 4202560 642514 1251 0 0 75088 88 0 0 25 0 5 0 596093913 2226823168 521415 33554432000 4194304 4417332 140735103880576 18446744073709551615 4358608 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3022.73
Current children cumulated vsize (KiB) 2174632

[startup+822.301 s]
/proc/loadavg: 8.06 8.06 7.95 10/186 29564
/proc/meminfo: memFree=15688792/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=2206164 CPUtime=3262.39 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) S 29527 29529 28898 0 -1 4202496 1921623 1251 0 0 325960 279 0 0 25 0 5 0 596093171 2259111936 529206 33554432000 4194304 4417332 140735103880576 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29529/statm: 551541 529206 181 55 0 549427 0
[pid=29529/tid=29537] ppid=29527 vsize=2206164 CPUtime=814.89 cores=1,3,5,7
/proc/29529/task/29537/stat : 29537 (plingeling) R 29527 29529 28898 0 -1 4202560 298140 1251 0 0 81443 46 0 0 25 0 5 0 596093913 2259111936 529206 33554432000 4194304 4417332 140735103880576 18446744073709551615 4263807 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29529/tid=29538] ppid=29527 vsize=2206164 CPUtime=813.46 cores=1,3,5,7
/proc/29529/task/29538/stat : 29538 (plingeling) R 29527 29529 28898 0 -1 4202560 295605 1251 0 0 81301 45 0 0 25 0 5 0 596093913 2259111936 529206 33554432000 4194304 4417332 140735103880576 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29529/tid=29539] ppid=29527 vsize=2206164 CPUtime=814.85 cores=1,3,5,7
/proc/29529/task/29539/stat : 29539 (plingeling) R 29527 29529 28898 0 -1 4202560 336341 1251 0 0 81435 50 0 0 25 0 5 0 596093913 2259111936 529206 33554432000 4194304 4417332 140735103880576 18446744073709551615 4254837 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29529/tid=29540] ppid=29527 vsize=2206164 CPUtime=811.75 cores=1,3,5,7
/proc/29529/task/29540/stat : 29540 (plingeling) R 29527 29529 28898 0 -1 4202560 687682 1251 0 0 81079 96 0 0 25 0 5 0 596093913 2259111936 529206 33554432000 4194304 4417332 140735103880576 18446744073709551615 4232680 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3262.39
Current children cumulated vsize (KiB) 2206164

[startup+882.301 s]
/proc/loadavg: 8.06 8.06 7.95 10/186 29566
/proc/meminfo: memFree=14695848/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=2357896 CPUtime=3502.06 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) S 29527 29529 28898 0 -1 4202496 2058421 1251 0 0 349907 299 0 0 25 0 5 0 596093171 2414485504 567170 33554432000 4194304 4417332 140735103880576 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29529/statm: 589474 567170 181 55 0 587360 0
[pid=29529/tid=29537] ppid=29527 vsize=2357896 CPUtime=874.89 cores=1,3,5,7
/proc/29529/task/29537/stat : 29537 (plingeling) R 29527 29529 28898 0 -1 4202560 306369 1251 0 0 87442 47 0 0 25 0 5 0 596093913 2414485504 567170 33554432000 4194304 4417332 140735103880576 18446744073709551615 4320304 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29529/tid=29538] ppid=29527 vsize=2357896 CPUtime=873.13 cores=1,3,5,7
/proc/29529/task/29538/stat : 29538 (plingeling) R 29527 29529 28898 0 -1 4202560 302092 1251 0 0 87267 46 0 0 25 0 5 0 596093913 2414485504 567170 33554432000 4194304 4417332 140735103880576 18446744073709551615 4288117 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29529/tid=29539] ppid=29527 vsize=2357896 CPUtime=874.84 cores=1,3,5,7
/proc/29529/task/29539/stat : 29539 (plingeling) R 29527 29529 28898 0 -1 4202560 336361 1251 0 0 87434 50 0 0 25 0 5 0 596093913 2414485504 567170 33554432000 4194304 4417332 140735103880576 18446744073709551615 4232612 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29529/tid=29540] ppid=29527 vsize=2357896 CPUtime=871.74 cores=1,3,5,7
/proc/29529/task/29540/stat : 29540 (plingeling) R 29527 29529 28898 0 -1 4202560 809744 1251 0 0 87060 114 0 0 25 0 5 0 596093913 2414485504 567170 33554432000 4194304 4417332 140735103880576 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3502.06
Current children cumulated vsize (KiB) 2357896

[startup+942.301 s]
/proc/loadavg: 8.13 8.08 7.97 10/186 29568
/proc/meminfo: memFree=15280564/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=2915040 CPUtime=3741.73 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) S 29527 29529 28898 0 -1 4202496 2243557 1251 0 0 373846 327 0 0 25 0 5 0 596093171 2985000960 706456 33554432000 4194304 4417332 140735103880576 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29529/statm: 728760 706456 181 55 0 726646 0
[pid=29529/tid=29537] ppid=29527 vsize=2915040 CPUtime=934.89 cores=1,3,5,7
/proc/29529/task/29537/stat : 29537 (plingeling) R 29527 29529 28898 0 -1 4202560 306369 1251 0 0 93442 47 0 0 25 0 5 0 596093913 2985000960 706456 33554432000 4194304 4417332 140735103880576 18446744073709551615 4252379 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29529/tid=29538] ppid=29527 vsize=2915040 CPUtime=932.81 cores=1,3,5,7
/proc/29529/task/29538/stat : 29538 (plingeling) R 29527 29529 28898 0 -1 4202560 302092 1251 0 0 93235 46 0 0 25 0 5 0 596093913 2985000960 706456 33554432000 4194304 4417332 140735103880576 18446744073709551615 4254855 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29529/tid=29539] ppid=29527 vsize=2915040 CPUtime=934.84 cores=1,3,5,7
/proc/29529/task/29539/stat : 29539 (plingeling) R 29527 29529 28898 0 -1 4202560 436251 1251 0 0 93420 64 0 0 25 0 5 0 596093913 2985000960 706456 33554432000 4194304 4417332 140735103880576 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29529/tid=29540] ppid=29527 vsize=2915040 CPUtime=931.74 cores=1,3,5,7
/proc/29529/task/29540/stat : 29540 (plingeling) R 29527 29529 28898 0 -1 4202560 894990 1251 0 0 93046 128 0 0 25 0 5 0 596093913 2985000960 706456 33554432000 4194304 4417332 140735103880576 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3741.73
Current children cumulated vsize (KiB) 2915040

[startup+1002.3 s]
/proc/loadavg: 8.12 8.08 7.97 10/186 29569
/proc/meminfo: memFree=15066592/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=2277028 CPUtime=3981.4 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) S 29527 29529 28898 0 -1 4202496 2277640 1251 0 0 397806 334 0 0 25 0 5 0 596093171 2331676672 546948 33554432000 4194304 4417332 140735103880576 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29529/statm: 569257 546948 181 55 0 567143 0
[pid=29529/tid=29537] ppid=29527 vsize=2277028 CPUtime=994.89 cores=1,3,5,7
/proc/29529/task/29537/stat : 29537 (plingeling) R 29527 29529 28898 0 -1 4202560 306369 1251 0 0 99442 47 0 0 25 0 5 0 596093913 2331676672 546948 33554432000 4194304 4417332 140735103880576 18446744073709551615 4364722 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29529/tid=29538] ppid=29527 vsize=2277028 CPUtime=992.48 cores=1,3,5,7
/proc/29529/task/29538/stat : 29538 (plingeling) R 29527 29529 28898 0 -1 4202560 302092 1251 0 0 99202 46 0 0 25 0 5 0 596093913 2331676672 546948 33554432000 4194304 4417332 140735103880576 18446744073709551615 4357164 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29529/tid=29539] ppid=29527 vsize=2277028 CPUtime=994.84 cores=1,3,5,7
/proc/29529/task/29539/stat : 29539 (plingeling) R 29527 29529 28898 0 -1 4202560 437541 1251 0 0 99418 66 0 0 25 0 5 0 596093913 2331676672 546948 33554432000 4194304 4417332 140735103880576 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29529/tid=29540] ppid=29527 vsize=2277028 CPUtime=991.74 cores=1,3,5,7
/proc/29529/task/29540/stat : 29540 (plingeling) R 29527 29529 28898 0 -1 4202560 927783 1251 0 0 99040 134 0 0 25 0 5 0 596093913 2331676672 546948 33554432000 4194304 4417332 140735103880576 18446744073709551615 4265403 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3981.4
Current children cumulated vsize (KiB) 2277028

[startup+1062.3 s]
/proc/loadavg: 8.17 8.09 7.98 10/186 29571
/proc/meminfo: memFree=15268764/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=2676164 CPUtime=4221.08 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) S 29527 29529 28898 0 -1 4202496 2411328 1251 0 0 421758 350 0 0 25 0 5 0 596093171 2740391936 646561 33554432000 4194304 4417332 140735103880576 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29529/statm: 669041 646561 181 55 0 666927 0
[pid=29529/tid=29537] ppid=29527 vsize=2676164 CPUtime=1054.9 cores=1,3,5,7
/proc/29529/task/29537/stat : 29537 (plingeling) R 29527 29529 28898 0 -1 4202560 370193 1251 0 0 105435 55 0 0 25 0 5 0 596093913 2740391936 646561 33554432000 4194304 4417332 140735103880576 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29529/tid=29538] ppid=29527 vsize=2676164 CPUtime=1052.16 cores=1,3,5,7
/proc/29529/task/29538/stat : 29538 (plingeling) R 29527 29529 28898 0 -1 4202560 302772 1251 0 0 105170 46 0 0 25 0 5 0 596093913 2740391936 646561 33554432000 4194304 4417332 140735103880576 18446744073709551615 4356771 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29529/tid=29539] ppid=29527 vsize=2676164 CPUtime=1054.84 cores=1,3,5,7
/proc/29529/task/29539/stat : 29539 (plingeling) R 29527 29529 28898 0 -1 4202560 441333 1251 0 0 105418 66 0 0 25 0 5 0 596093913 2740391936 646561 33554432000 4194304 4417332 140735103880576 18446744073709551615 4232801 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29529/tid=29540] ppid=29527 vsize=2676164 CPUtime=1051.72 cores=1,3,5,7
/proc/29529/task/29540/stat : 29540 (plingeling) R 29527 29529 28898 0 -1 4202560 993175 1251 0 0 105031 141 0 0 25 0 5 0 596093913 2740391936 646561 33554432000 4194304 4417332 140735103880576 18446744073709551615 4365065 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4221.08
Current children cumulated vsize (KiB) 2676164

[startup+1122.3 s]
/proc/loadavg: 6.75 7.77 7.88 5/176 29590
/proc/meminfo: memFree=16196600/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=3438744 CPUtime=4460.75 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) S 29527 29529 28898 0 -1 4202496 2602042 1251 0 0 445697 378 0 0 25 0 5 0 596093171 3521273856 837206 33554432000 4194304 4417332 140735103880576 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29529/statm: 859686 837206 181 55 0 857572 0
[pid=29529/tid=29537] ppid=29527 vsize=3438744 CPUtime=1114.9 cores=1,3,5,7
/proc/29529/task/29537/stat : 29537 (plingeling) R 29527 29529 28898 0 -1 4202560 370193 1251 0 0 111435 55 0 0 25 0 5 0 596093913 3521273856 837206 33554432000 4194304 4417332 140735103880576 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29529/tid=29538] ppid=29527 vsize=3438744 CPUtime=1111.83 cores=1,3,5,7
/proc/29529/task/29538/stat : 29538 (plingeling) R 29527 29529 28898 0 -1 4202560 341553 1251 0 0 111132 51 0 0 25 0 5 0 596093913 3521273856 837206 33554432000 4194304 4417332 140735103880576 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29529/tid=29539] ppid=29527 vsize=3438744 CPUtime=1114.84 cores=1,3,5,7
/proc/29529/task/29539/stat : 29539 (plingeling) R 29527 29529 28898 0 -1 4202560 520124 1251 0 0 111407 77 0 0 25 0 5 0 596093913 3521273856 837206 33554432000 4194304 4417332 140735103880576 18446744073709551615 4265992 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29529/tid=29540] ppid=29527 vsize=3438744 CPUtime=1111.73 cores=1,3,5,7
/proc/29529/task/29540/stat : 29540 (plingeling) R 29527 29529 28898 0 -1 4202560 1066317 1251 0 0 111020 153 0 0 25 0 5 0 596093913 3521273856 837206 33554432000 4194304 4417332 140735103880576 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4460.75
Current children cumulated vsize (KiB) 3438744

[startup+1182.3 s]
/proc/loadavg: 5.01 7.08 7.63 5/176 29592
/proc/meminfo: memFree=15955256/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=3679820 CPUtime=4700.44 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) S 29527 29529 28898 0 -1 4202496 2789254 1251 0 0 469633 411 0 0 25 0 5 0 596093171 3768135680 897485 33554432000 4194304 4417332 140735103880576 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29529/statm: 919955 897485 181 55 0 917841 0
[pid=29529/tid=29537] ppid=29527 vsize=3679820 CPUtime=1174.9 cores=1,3,5,7
/proc/29529/task/29537/stat : 29537 (plingeling) R 29527 29529 28898 0 -1 4202560 400018 1251 0 0 117432 58 0 0 25 0 5 0 596093913 3768135680 897485 33554432000 4194304 4417332 140735103880576 18446744073709551615 4357458 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29529/tid=29538] ppid=29527 vsize=3679820 CPUtime=1171.52 cores=1,3,5,7
/proc/29529/task/29538/stat : 29538 (plingeling) R 29527 29529 28898 0 -1 4202560 387143 1251 0 0 117094 58 0 0 25 0 5 0 596093913 3768135680 897485 33554432000 4194304 4417332 140735103880576 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29529/tid=29539] ppid=29527 vsize=3679820 CPUtime=1174.84 cores=1,3,5,7
/proc/29529/task/29539/stat : 29539 (plingeling) R 29527 29529 28898 0 -1 4202560 576102 1251 0 0 117394 90 0 0 25 0 5 0 596093913 3768135680 897485 33554432000 4194304 4417332 140735103880576 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29529/tid=29540] ppid=29527 vsize=3679820 CPUtime=1171.73 cores=1,3,5,7
/proc/29529/task/29540/stat : 29540 (plingeling) R 29527 29529 28898 0 -1 4202560 1122136 1251 0 0 117010 163 0 0 25 0 5 0 596093913 3768135680 897485 33554432000 4194304 4417332 140735103880576 18446744073709551615 4364346 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4700.44
Current children cumulated vsize (KiB) 3679820



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1207.3 s]
/proc/loadavg: 4.66 6.83 7.54 5/176 29593
/proc/meminfo: memFree=15714940/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=3918872 CPUtime=4800.32 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) S 29527 29529 28898 0 -1 4202496 2849165 1251 0 0 479611 421 0 0 25 0 5 0 596093171 4012924928 957248 33554432000 4194304 4417332 140735103880576 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29529/statm: 979718 957248 181 55 0 977604 0
[pid=29529/tid=29537] ppid=29527 vsize=3918872 CPUtime=1199.91 cores=1,3,5,7
/proc/29529/task/29537/stat : 29537 (plingeling) R 29527 29529 28898 0 -1 4202560 401984 1251 0 0 119933 58 0 0 25 0 5 0 596093913 4012924928 957248 33554432000 4194304 4417332 140735103880576 18446744073709551615 4353461 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29529/tid=29538] ppid=29527 vsize=3918872 CPUtime=1196.38 cores=1,3,5,7
/proc/29529/task/29538/stat : 29538 (plingeling) R 29527 29529 28898 0 -1 4202560 395941 1251 0 0 119579 59 0 0 25 0 5 0 596093913 4012924928 957248 33554432000 4194304 4417332 140735103880576 18446744073709551615 4250936 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29529/tid=29539] ppid=29527 vsize=3918872 CPUtime=1199.85 cores=1,3,5,7
/proc/29529/task/29539/stat : 29539 (plingeling) R 29527 29529 28898 0 -1 4202560 576102 1251 0 0 119895 90 0 0 25 0 5 0 596093913 4012924928 957248 33554432000 4194304 4417332 140735103880576 18446744073709551615 4356756 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29529/tid=29540] ppid=29527 vsize=3918872 CPUtime=1196.74 cores=1,3,5,7
/proc/29529/task/29540/stat : 29540 (plingeling) R 29527 29529 28898 0 -1 4202560 1171283 1251 0 0 119502 172 0 0 25 0 5 0 596093913 4012924928 957248 33554432000 4194304 4417332 140735103880576 18446744073709551615 4302950 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.32
Current children cumulated vsize (KiB) 3918872

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1207.41 s]
/proc/loadavg: 4.66 6.83 7.54 5/176 29593
/proc/meminfo: memFree=15714940/32950928 swapFree=67111364/67111528
[pid=29529] ppid=29527 vsize=0 CPUtime=4800.44 cores=1,3,5,7
/proc/29529/stat : 29529 (plingeling) Z 29527 29529 28898 0 -1 4203548 2849165 1251 0 0 479613 431 0 0 18 0 2 0 596093171 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29529/statm: 0 0 0 0 0 0 0
[pid=29529/tid=29538] ppid=29527 vsize=0 CPUtime=1196.48 cores=1,3,5,7
/proc/29529/task/29538/stat : 29538 (plingeling) R 29527 29529 28898 0 -1 4203588 395941 1251 0 0 119579 69 0 0 25 0 2 0 596093913 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.44
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1207.46
CPU time (s): 4800.5
CPU user time (s): 4796.14
CPU system time (s): 4.36833
CPU usage (%): 397.569
Max. virtual memory (cumulated for all children) (KiB): 3918872

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.14
system time used= 4.36834
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2850416
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 187
involuntary context switches= 39179

runsolver used 1.84672 second user time and 4.74428 second system time

The end

Launcher Data 

Begin job on node113 at 2011-04-29 08:30:25
IDJOB=3285251
IDBENCH=82374
IDSOLVER=1605
FILE ID=node113/3285251-1304058621
RUNJOBID= node113-1304056091-28917
PBS_JOBID= 13170432
Free space on /tmp= 71024 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/application/rintanen/SATPlanning/transport-transport-city-sequential-35nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.040-NOTKNOWN.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3285251-1304058621/watcher-3285251-1304058621 -o /tmp/evaluation-result-3285251-1304058621/solver-3285251-1304058621 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3285251-1304058621.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= ce0d180356398dfbcfd2bbbb1cb3f9e5
RANDOM SEED=1875226145

node113.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      19138552 kB
Buffers:       1744012 kB
Cached:       11010260 kB
SwapCached:        164 kB
Active:        6918596 kB
Inactive:      6299816 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      19138552 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:          134024 kB
Writeback:           0 kB
AnonPages:      463336 kB
Mapped:          15344 kB
Slab:           529356 kB
PageTables:       5556 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   655340 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71036 MiB
End job on node113 at 2011-04-29 08:50:34