Trace number 3371042

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.39 1203.14

General information on the benchmark

NameSAT03/handmade/stanion/hwb/
hwb-n28-02-S818962541.shuffled-as.sat03-1628.cnf
MD5SUMd705043763df769aab1e385bbbc2b11e
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables184
Number of clauses896
Sum of the clauses size3042
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 214
Number of clauses of size 3514
Number of clauses of size 4368
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371042-1305430150/watcher-3371042-1305430150 -o /tmp/evaluation-result-3371042-1305430150/solver-3371042-1305430150 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371042-1305430150.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.89 7.85 7.93 3/313 18750
/proc/meminfo: memFree=22906976/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=8728 CPUtime=0 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 216 0 0 0 0 0 0 0 21 0 1 0 733245470 8937472 146 33554432000 4194304 4417452 140737305354848 18446744073709551615 252458065664 0 0 4096 0 18446744071562362840 0 0 17 4 0 0 0
/proc/18748/statm: 2182 146 122 55 0 68 0
[pid=18748/tid=18761] ppid=18746 vsize=50164 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 37 1254 0 0 0 0 0 0 22 0 5 0 733245471 51367936 339 33554432000 4194304 4417452 140737305354848 18446744073709551615 4253855 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=50164 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 0 1254 0 0 0 0 0 0 23 0 5 0 733245471 51367936 343 33554432000 4194304 4417452 140737305354848 18446744073709551615 252458122225 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=50164 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 17 1254 0 0 0 0 0 0 23 0 5 0 733245471 51367936 343 33554432000 4194304 4417452 140737305354848 18446744073709551615 4327349 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=50164 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 20 1254 0 0 0 0 0 0 23 0 5 0 733245471 51367936 343 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 4 0 0 0

[startup+0.0956221 s]
/proc/loadavg: 6.89 7.85 7.93 3/313 18750
/proc/meminfo: memFree=22906976/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=52312 CPUtime=0.34 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 1050 1254 0 0 34 0 0 0 21 0 5 0 733245470 53567488 901 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 13078 901 180 55 0 10964 0
[pid=18748/tid=18761] ppid=18746 vsize=52312 CPUtime=0.08 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 163 1254 0 0 8 0 0 0 22 0 5 0 733245471 53567488 901 33554432000 4194304 4417452 140737305354848 18446744073709551615 4319944 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=52312 CPUtime=0.08 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 170 1254 0 0 8 0 0 0 24 0 5 0 733245471 53567488 901 33554432000 4194304 4417452 140737305354848 18446744073709551615 4252155 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=52312 CPUtime=0.08 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 209 1254 0 0 8 0 0 0 24 0 5 0 733245471 53567488 901 33554432000 4194304 4417452 140737305354848 18446744073709551615 4366486 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=52312 CPUtime=0.08 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 168 1254 0 0 8 0 0 0 23 0 5 0 733245471 53567488 901 33554432000 4194304 4417452 140737305354848 18446744073709551615 4365188 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 0.34
Current children cumulated vsize (KiB) 52312

[startup+0.100621 s]
/proc/loadavg: 6.89 7.85 7.93 3/313 18750
/proc/meminfo: memFree=22906976/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=52608 CPUtime=0.36 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 1124 1254 0 0 36 0 0 0 21 0 5 0 733245470 53870592 975 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 13152 975 180 55 0 11038 0
[pid=18748/tid=18761] ppid=18746 vsize=52608 CPUtime=0.09 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 182 1254 0 0 9 0 0 0 23 0 5 0 733245471 53870592 975 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364614 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=52608 CPUtime=0.08 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 170 1254 0 0 8 0 0 0 24 0 5 0 733245471 53870592 975 33554432000 4194304 4417452 140737305354848 18446744073709551615 4258491 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=52608 CPUtime=0.08 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 264 1254 0 0 8 0 0 0 24 0 5 0 733245471 53870592 975 33554432000 4194304 4417452 140737305354848 18446744073709551615 4366294 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=52608 CPUtime=0.09 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 168 1254 0 0 9 0 0 0 23 0 5 0 733245471 53870592 975 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364784 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 0.36
Current children cumulated vsize (KiB) 52608

[startup+0.300585 s]
/proc/loadavg: 6.89 7.85 7.93 3/313 18750
/proc/meminfo: memFree=22906976/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=54084 CPUtime=1.15 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 2741 1254 0 0 115 0 0 0 21 0 5 0 733245470 55382016 1352 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 13521 1352 180 55 0 11407 0
[pid=18748/tid=18761] ppid=18746 vsize=54084 CPUtime=0.29 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 570 1254 0 0 29 0 0 0 23 0 5 0 733245471 55382016 1352 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=54084 CPUtime=0.28 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 551 1254 0 0 28 0 0 0 25 0 5 0 733245471 55382016 1352 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364567 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=54084 CPUtime=0.28 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 668 1254 0 0 28 0 0 0 25 0 5 0 733245471 55382016 1352 33554432000 4194304 4417452 140737305354848 18446744073709551615 4306245 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=54084 CPUtime=0.29 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 612 1254 0 0 29 0 0 0 25 0 5 0 733245471 55382016 1352 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1.15
Current children cumulated vsize (KiB) 54084

[startup+0.700512 s]
/proc/loadavg: 6.89 7.85 7.93 3/313 18750
/proc/meminfo: memFree=22906976/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=57256 CPUtime=2.75 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 5106 1254 0 0 275 0 0 0 21 0 5 0 733245470 58630144 2149 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 14314 2149 180 55 0 12200 0
[pid=18748/tid=18761] ppid=18746 vsize=57256 CPUtime=0.69 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 1236 1254 0 0 69 0 0 0 25 0 5 0 733245471 58630144 2149 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=57256 CPUtime=0.68 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 1220 1254 0 0 68 0 0 0 25 0 5 0 733245471 58630144 2149 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=57256 CPUtime=0.68 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 1357 1254 0 0 68 0 0 0 25 0 5 0 733245471 58630144 2149 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=57256 CPUtime=0.69 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 953 1254 0 0 69 0 0 0 25 0 5 0 733245471 58630144 2149 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2.75
Current children cumulated vsize (KiB) 57256

[startup+1.50036 s]
/proc/loadavg: 6.89 7.85 7.93 10/323 18770
/proc/meminfo: memFree=22886936/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=59788 CPUtime=5.94 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 9893 1254 0 0 593 1 0 0 21 0 5 0 733245470 61222912 2750 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 14947 2750 180 55 0 12833 0
[pid=18748/tid=18761] ppid=18746 vsize=59788 CPUtime=1.48 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 2602 1254 0 0 148 0 0 0 25 0 5 0 733245471 61222912 2750 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=59788 CPUtime=1.46 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 2253 1254 0 0 146 0 0 0 25 0 5 0 733245471 61222912 2750 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=59788 CPUtime=1.48 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 2786 1254 0 0 148 0 0 0 25 0 5 0 733245471 61222912 2750 33554432000 4194304 4417452 140737305354848 18446744073709551615 4366486 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=59788 CPUtime=1.48 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 1912 1254 0 0 148 0 0 0 25 0 5 0 733245471 61222912 2750 33554432000 4194304 4417452 140737305354848 18446744073709551615 4366698 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 5.94
Current children cumulated vsize (KiB) 59788

[startup+3.10107 s]
/proc/loadavg: 6.89 7.85 7.93 10/323 18770
/proc/meminfo: memFree=22880240/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=64088 CPUtime=12.32 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 16997 1254 0 0 1231 1 0 0 21 0 5 0 733245470 65626112 3832 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 16022 3832 180 55 0 13908 0
[pid=18748/tid=18761] ppid=18746 vsize=64088 CPUtime=3.08 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 4850 1254 0 0 308 0 0 0 25 0 5 0 733245471 65626112 3832 33554432000 4194304 4417452 140737305354848 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=64088 CPUtime=3.05 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 3983 1254 0 0 305 0 0 0 25 0 5 0 733245471 65626112 3832 33554432000 4194304 4417452 140737305354848 18446744073709551615 4354515 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=64088 CPUtime=3.08 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 4434 1254 0 0 308 0 0 0 25 0 5 0 733245471 65626112 3832 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=64088 CPUtime=3.09 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 3390 1254 0 0 309 0 0 0 25 0 5 0 733245471 65626112 3832 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 12.32
Current children cumulated vsize (KiB) 64088

[startup+6.30047 s]
/proc/loadavg: 6.98 7.85 7.93 10/323 18770
/proc/meminfo: memFree=22863448/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=72372 CPUtime=25.09 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 29414 1254 0 0 2506 3 0 0 21 0 5 0 733245470 74108928 5853 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 18093 5853 180 55 0 15979 0
[pid=18748/tid=18761] ppid=18746 vsize=72372 CPUtime=6.28 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 7616 1254 0 0 628 0 0 0 25 0 5 0 733245471 74108928 5853 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=72372 CPUtime=6.22 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 6704 1254 0 0 621 1 0 0 25 0 5 0 733245471 74108928 5853 33554432000 4194304 4417452 140737305354848 18446744073709551615 4250464 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=72372 CPUtime=6.28 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 7970 1254 0 0 627 1 0 0 25 0 5 0 733245471 74108928 5853 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=72372 CPUtime=6.28 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 6784 1254 0 0 628 0 0 0 25 0 5 0 733245471 74108928 5853 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 25.09
Current children cumulated vsize (KiB) 72372

[startup+12.7003 s]
/proc/loadavg: 7.06 7.85 7.93 10/323 18770
/proc/meminfo: memFree=22848500/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=77200 CPUtime=50.63 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 48247 1254 0 0 5057 6 0 0 21 0 5 0 733245470 79052800 7103 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 19300 7103 180 55 0 17186 0
[pid=18748/tid=18761] ppid=18746 vsize=77200 CPUtime=12.68 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 13148 1254 0 0 1267 1 0 0 25 0 5 0 733245471 79052800 7103 33554432000 4194304 4417452 140737305354848 18446744073709551615 4236956 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=77200 CPUtime=12.55 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 11530 1254 0 0 1253 2 0 0 25 0 5 0 733245471 79052800 7103 33554432000 4194304 4417452 140737305354848 18446744073709551615 4199406 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=77200 CPUtime=12.68 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 12793 1254 0 0 1267 1 0 0 25 0 5 0 733245471 79052800 7103 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=77200 CPUtime=12.68 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 10436 1254 0 0 1268 0 0 0 25 0 5 0 733245471 79052800 7103 33554432000 4194304 4417452 140737305354848 18446744073709551615 4365092 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 50.63
Current children cumulated vsize (KiB) 77200

[startup+25.5009 s]
/proc/loadavg: 7.27 7.86 7.93 10/323 18770
/proc/meminfo: memFree=22825036/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=89304 CPUtime=101.7 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 80965 1254 0 0 10159 11 0 0 21 0 5 0 733245470 91447296 10154 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 22326 10154 180 55 0 20212 0
[pid=18748/tid=18761] ppid=18746 vsize=89304 CPUtime=25.48 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 22225 1254 0 0 2546 2 0 0 25 0 5 0 733245471 91447296 10154 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=89304 CPUtime=25.22 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 19505 1254 0 0 2519 3 0 0 25 0 5 0 733245471 91447296 10154 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=89304 CPUtime=25.48 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 21371 1254 0 0 2545 3 0 0 25 0 5 0 733245471 91447296 10154 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=89304 CPUtime=25.49 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 17524 1254 0 0 2547 2 0 0 25 0 5 0 733245471 91447296 10154 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 101.7
Current children cumulated vsize (KiB) 89304

[startup+51.1011 s]
/proc/loadavg: 7.60 7.89 7.94 9/323 18771
/proc/meminfo: memFree=22806188/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=99092 CPUtime=203.84 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 120541 1254 0 0 20368 16 0 0 21 0 5 0 733245470 101470208 12597 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 24773 12597 180 55 0 22659 0
[pid=18748/tid=18761] ppid=18746 vsize=99092 CPUtime=51.08 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 33820 1254 0 0 5105 3 0 0 25 0 5 0 733245471 101470208 12597 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=99092 CPUtime=50.56 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 30408 1254 0 0 5051 5 0 0 25 0 5 0 733245471 101470208 12597 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=99092 CPUtime=51.08 cores=0,2,4,6

################
# More data... #
################

[pid=18748/tid=18766] ppid=18746 vsize=448016 CPUtime=754.57 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 127128 1254 0 0 75435 22 0 0 25 0 5 0 733245471 458768384 99807 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364885 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=448016 CPUtime=762.31 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 133731 1254 0 0 76211 20 0 0 25 0 5 0 733245471 458768384 99807 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=448016 CPUtime=762.32 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 137900 1254 0 0 76213 19 0 0 25 0 5 0 733245471 458768384 99807 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3041.55
Current children cumulated vsize (KiB) 448016

[startup+822.301 s]
/proc/loadavg: 8.16 8.12 8.04 10/323 18795
/proc/meminfo: memFree=22108308/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=538236 CPUtime=3280.91 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 550347 1254 0 0 328007 84 0 0 21 0 5 0 733245470 551153664 122362 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 134559 122362 180 55 0 132445 0
[pid=18748/tid=18761] ppid=18746 vsize=538236 CPUtime=822.31 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 136710 1254 0 0 82213 18 0 0 25 0 5 0 733245471 551153664 122362 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=538236 CPUtime=813.96 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 127128 1254 0 0 81374 22 0 0 25 0 5 0 733245471 551153664 122362 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=538236 CPUtime=822.3 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 142518 1254 0 0 82208 22 0 0 25 0 5 0 733245471 551153664 122362 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=538236 CPUtime=822.31 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 143651 1254 0 0 82210 21 0 0 25 0 5 0 733245471 551153664 122362 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3280.91
Current children cumulated vsize (KiB) 538236

[startup+882.301 s]
/proc/loadavg: 8.16 8.13 8.04 10/323 18796
/proc/meminfo: memFree=22048144/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=576656 CPUtime=3520.3 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 559952 1254 0 0 351944 86 0 0 21 0 5 0 733245470 590495744 131967 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 144164 131967 180 55 0 142050 0
[pid=18748/tid=18761] ppid=18746 vsize=576656 CPUtime=882.31 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 136710 1254 0 0 88213 18 0 0 25 0 5 0 733245471 590495744 131967 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=576656 CPUtime=873.34 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 136733 1254 0 0 87311 23 0 0 25 0 5 0 733245471 590495744 131967 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=576656 CPUtime=882.3 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 142518 1254 0 0 88208 22 0 0 25 0 5 0 733245471 590495744 131967 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=576656 CPUtime=882.31 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 143651 1254 0 0 88210 21 0 0 25 0 5 0 733245471 590495744 131967 33554432000 4194304 4417452 140737305354848 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3520.3
Current children cumulated vsize (KiB) 576656

[startup+942.301 s]
/proc/loadavg: 8.09 8.11 8.04 10/323 18798
/proc/meminfo: memFree=22048988/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=565204 CPUtime=3759.71 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 580240 1254 0 0 375882 89 0 0 21 0 5 0 733245470 578768896 129104 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 141301 129104 180 55 0 139187 0
[pid=18748/tid=18761] ppid=18746 vsize=565204 CPUtime=942.31 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 136710 1254 0 0 94213 18 0 0 25 0 5 0 733245471 578768896 129104 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=565204 CPUtime=932.74 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 148189 1254 0 0 93249 25 0 0 25 0 5 0 733245471 578768896 129104 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=565204 CPUtime=942.31 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 144423 1254 0 0 94208 23 0 0 25 0 5 0 733245471 578768896 129104 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=565204 CPUtime=942.32 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 150578 1254 0 0 94210 22 0 0 25 0 5 0 733245471 578768896 129104 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3759.71
Current children cumulated vsize (KiB) 565204

[startup+1002.3 s]
/proc/loadavg: 8.11 8.11 8.04 10/323 18799
/proc/meminfo: memFree=21891492/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=632296 CPUtime=3999.1 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 597013 1254 0 0 399817 93 0 0 21 0 5 0 733245470 647471104 145877 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 158074 145877 180 55 0 155960 0
[pid=18748/tid=18761] ppid=18746 vsize=632296 CPUtime=1002.32 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 144902 1254 0 0 100212 20 0 0 25 0 5 0 733245471 647471104 145877 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=632296 CPUtime=992.12 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 148578 1254 0 0 99187 25 0 0 25 0 5 0 733245471 647471104 145877 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=632296 CPUtime=1002.31 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 152615 1254 0 0 100207 24 0 0 25 0 5 0 733245471 647471104 145877 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=632296 CPUtime=1002.32 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 150578 1254 0 0 100210 22 0 0 25 0 5 0 733245471 647471104 145877 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3999.1
Current children cumulated vsize (KiB) 632296

[startup+1062.3 s]
/proc/loadavg: 8.12 8.10 8.04 10/323 18801
/proc/meminfo: memFree=21878212/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=630940 CPUtime=4238.5 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 614919 1254 0 0 423755 95 0 0 21 0 5 0 733245470 646082560 145538 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 157735 145538 180 55 0 155621 0
[pid=18748/tid=18761] ppid=18746 vsize=630940 CPUtime=1062.32 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 145574 1254 0 0 106212 20 0 0 25 0 5 0 733245471 646082560 145538 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=630940 CPUtime=1051.52 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 148578 1254 0 0 105127 25 0 0 25 0 5 0 733245471 646082560 145538 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=630940 CPUtime=1062.31 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 152615 1254 0 0 106207 24 0 0 25 0 5 0 733245471 646082560 145538 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=630940 CPUtime=1062.31 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 167812 1254 0 0 106207 24 0 0 25 0 5 0 733245471 646082560 145538 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4238.5
Current children cumulated vsize (KiB) 630940

[startup+1122.3 s]
/proc/loadavg: 8.10 8.10 8.04 10/323 18803
/proc/meminfo: memFree=21879048/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=601280 CPUtime=4477.91 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 615696 1254 0 0 447695 96 0 0 21 0 5 0 733245470 615710720 138123 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 150320 138123 180 55 0 148206 0
[pid=18748/tid=18761] ppid=18746 vsize=601280 CPUtime=1122.32 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 145574 1254 0 0 112212 20 0 0 25 0 5 0 733245471 615710720 138123 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364583 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=601280 CPUtime=1110.91 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 149355 1254 0 0 111066 25 0 0 25 0 5 0 733245471 615710720 138123 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=601280 CPUtime=1122.31 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 152615 1254 0 0 112207 24 0 0 25 0 5 0 733245471 615710720 138123 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=601280 CPUtime=1122.32 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 167812 1254 0 0 112208 24 0 0 25 0 5 0 733245471 615710720 138123 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4477.91
Current children cumulated vsize (KiB) 601280

[startup+1182.3 s]
/proc/loadavg: 8.10 8.09 8.04 10/323 18804
/proc/meminfo: memFree=21845676/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=664372 CPUtime=4717.3 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 631469 1254 0 0 471632 98 0 0 21 0 5 0 733245470 680316928 153896 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 166093 153896 180 55 0 163979 0
[pid=18748/tid=18761] ppid=18746 vsize=664372 CPUtime=1182.32 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 145574 1254 0 0 118212 20 0 0 25 0 5 0 733245471 680316928 153896 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=664372 CPUtime=1170.31 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 160089 1254 0 0 117003 28 0 0 25 0 5 0 733245471 680316928 153896 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=664372 CPUtime=1182.31 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 152615 1254 0 0 118207 24 0 0 25 0 5 0 733245471 680316928 153896 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=664372 CPUtime=1182.32 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 172851 1254 0 0 118207 25 0 0 25 0 5 0 733245471 680316928 153896 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4717.3
Current children cumulated vsize (KiB) 664372



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.1 s]
/proc/loadavg: 8.07 8.08 8.04 9/323 18805
/proc/meminfo: memFree=21848888/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=624004 CPUtime=4800.31 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 661195 1254 0 0 479930 101 0 0 21 0 5 0 733245470 638980096 143836 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 156001 143836 180 55 0 153887 0
[pid=18748/tid=18761] ppid=18746 vsize=624004 CPUtime=1203.13 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 161356 1254 0 0 120292 21 0 0 25 0 5 0 733245471 638980096 143836 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=624004 CPUtime=1190.9 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 160089 1254 0 0 119062 28 0 0 25 0 5 0 733245471 638980096 143836 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=624004 CPUtime=1203.13 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 166559 1254 0 0 120287 26 0 0 25 0 5 0 733245471 638980096 143836 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=624004 CPUtime=1203.14 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 172851 1254 0 0 120289 25 0 0 25 0 5 0 733245471 638980096 143836 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.31
Current children cumulated vsize (KiB) 624004

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.1 s]
/proc/loadavg: 8.07 8.08 8.04 9/323 18805
/proc/meminfo: memFree=21848888/32951124 swapFree=59305796/67111528
[pid=18748] ppid=18746 vsize=624004 CPUtime=4800.31 cores=0,2,4,6
/proc/18748/stat : 18748 (plingeling) S 18746 18748 11662 0 -1 4202496 661195 1254 0 0 479930 101 0 0 21 0 5 0 733245470 638980096 143836 33554432000 4194304 4417452 140737305354848 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18748/statm: 156001 143836 180 55 0 153887 0
[pid=18748/tid=18761] ppid=18746 vsize=624004 CPUtime=1203.13 cores=0,2,4,6
/proc/18748/task/18761/stat : 18761 (plingeling) R 18746 18748 11662 0 -1 4202560 161356 1254 0 0 120292 21 0 0 25 0 5 0 733245471 638980096 143836 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18748/tid=18766] ppid=18746 vsize=624004 CPUtime=1190.9 cores=0,2,4,6
/proc/18748/task/18766/stat : 18766 (plingeling) R 18746 18748 11662 0 -1 4202560 160089 1254 0 0 119062 28 0 0 25 0 5 0 733245471 638980096 143836 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18748/tid=18767] ppid=18746 vsize=624004 CPUtime=1203.13 cores=0,2,4,6
/proc/18748/task/18767/stat : 18767 (plingeling) R 18746 18748 11662 0 -1 4202560 166559 1254 0 0 120287 26 0 0 25 0 5 0 733245471 638980096 143836 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18748/tid=18768] ppid=18746 vsize=624004 CPUtime=1203.14 cores=0,2,4,6
/proc/18748/task/18768/stat : 18768 (plingeling) R 18746 18748 11662 0 -1 4202560 172851 1254 0 0 120289 25 0 0 25 0 5 0 733245471 638980096 143836 33554432000 4194304 4417452 140737305354848 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.31
Current children cumulated vsize (KiB) 624004

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.14
CPU time (s): 4800.39
CPU user time (s): 4799.34
CPU system time (s): 1.04984
CPU usage (%): 398.988
Max. virtual memory (cumulated for all children) (KiB): 678036

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4799.34
system time used= 1.04984
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 662449
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 628
involuntary context switches= 27725

runsolver used 3.51047 second user time and 8.5357 second system time

The end

Launcher Data 

Begin job on node111 at 2011-05-15 05:29:10
IDJOB=3371042
IDBENCH=84018
IDSOLVER=1848
FILE ID=node111/3371042-1305430150
RUNJOBID= node111-1305430150-18731
PBS_JOBID= 13324596
Free space on /tmp= 73732 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT03/handmade/stanion/hwb/hwb-n28-02-S818962541.shuffled-as.sat03-1628.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371042-1305430150/watcher-3371042-1305430150 -o /tmp/evaluation-result-3371042-1305430150/solver-3371042-1305430150 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371042-1305430150.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= d705043763df769aab1e385bbbc2b11e
RANDOM SEED=1292092606

node111.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5428.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      22907644 kB
Buffers:        273284 kB
Cached:        6526212 kB
SwapCached:      13720 kB
Active:         943836 kB
Inactive:      8884136 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      22907644 kB
SwapTotal:    67111528 kB
SwapFree:     59305796 kB
Dirty:             496 kB
Writeback:           0 kB
AnonPages:     3024108 kB
Mapped:          12244 kB
Slab:           115780 kB
PageTables:      36620 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 53315544 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73732 MiB
End job on node111 at 2011-05-15 05:49:15