Trace number 3309282

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 40000.3 5004

General information on the benchmark

Nameindustrial/IBM_FV_2002_13_rule_1/
SAT_dat.k45.cnf
MD5SUM8a76c8add64173ae39149aae572a02d9
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark168.225
Satisfiable
(Un)Satisfiability was proved
Number of variables37432
Number of clauses147345
Sum of the clauses size401667
Maximum clause length37
Minimum clause length1
Number of clauses of size 1603
Number of clauses of size 269621
Number of clauses of size 371450
Number of clauses of size 4266
Number of clauses of size 5505
Number of clauses of size over 54900

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3309282-1303992406/watcher-3309282-1303992406 -o /tmp/evaluation-result-3309282-1303992406/solver-3309282-1303992406 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3309282-1303992406.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.36 7.88 7.95 2/170 22582
/proc/meminfo: memFree=14603792/32950928 swapFree=67111528/67111528
[pid=22582] ppid=22580 vsize=8728 CPUtime=0 cores=0-7
/proc/22582/stat : 22582 (plingeling) R 22580 22582 22318 0 -1 4202496 256 1261 0 0 0 0 0 0 25 0 1 0 589471256 8937472 184 33554432000 4194304 4417332 140736021182016 18446744073709551615 218880564992 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/22582/statm: 2182 184 137 55 0 68 0

[startup+0.0842381 s]
/proc/loadavg: 7.36 7.88 7.95 2/170 22582
/proc/meminfo: memFree=14603792/32950928 swapFree=67111528/67111528
[pid=22582] ppid=22580 vsize=23204 CPUtime=0.07 cores=0-7
/proc/22582/stat : 22582 (plingeling) R 22580 22582 22318 0 -1 4202496 3896 1261 0 0 7 0 0 0 25 0 1 0 589471256 23760896 3824 33554432000 4194304 4417332 140736021182016 18446744073709551615 4228002 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/22582/statm: 5801 3824 148 55 0 3687 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 23204

[startup+0.100245 s]
/proc/loadavg: 7.36 7.88 7.95 2/170 22582
/proc/meminfo: memFree=14603792/32950928 swapFree=67111528/67111528
[pid=22582] ppid=22580 vsize=28220 CPUtime=0.09 cores=0-7
/proc/22582/stat : 22582 (plingeling) R 22580 22582 22318 0 -1 4202496 5152 1261 0 0 9 0 0 0 25 0 1 0 589471256 28897280 5078 33554432000 4194304 4417332 140736021182016 18446744073709551615 4259140 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/22582/statm: 7055 5078 148 55 0 4941 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 28220

[startup+0.3002 s]
/proc/loadavg: 7.36 7.88 7.95 2/170 22582
/proc/meminfo: memFree=14603792/32950928 swapFree=67111528/67111528
[pid=22582] ppid=22580 vsize=63596 CPUtime=0.29 cores=0-7
/proc/22582/stat : 22582 (plingeling) R 22580 22582 22318 0 -1 4202496 13996 1261 0 0 28 1 0 0 25 0 1 0 589471256 65122304 13922 33554432000 4194304 4417332 140736021182016 18446744073709551615 4231303 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/22582/statm: 15899 13922 148 55 0 13785 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 63596

[startup+1.33294 s]
/proc/loadavg: 7.36 7.88 7.95 2/170 22582
/proc/meminfo: memFree=14603792/32950928 swapFree=67111528/67111528
[pid=22582] ppid=22580 vsize=346572 CPUtime=7.72 cores=0-7
/proc/22582/stat : 22582 (plingeling) S 22580 22582 22318 0 -1 4202496 47063 1261 0 0 767 5 0 0 18 0 9 0 589471256 354889728 19072 33554432000 4194304 4417332 140736021182016 18446744073709551615 218892367845 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/22582/statm: 86643 19072 176 55 0 84529 0
Current children cumulated CPU time (s) 7.72
Current children cumulated vsize (KiB) 346572

[startup+1.50092 s]
/proc/loadavg: 7.36 7.88 7.95 2/170 22582
/proc/meminfo: memFree=14603792/32950928 swapFree=67111528/67111528
[pid=22582] ppid=22580 vsize=346352 CPUtime=9.06 cores=0-7
/proc/22582/stat : 22582 (plingeling) S 22580 22582 22318 0 -1 4202496 48226 1261 0 0 901 5 0 0 18 0 9 0 589471256 354664448 18907 33554432000 4194304 4417332 140736021182016 18446744073709551615 218892367845 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/22582/statm: 86588 18907 177 55 0 84474 0
Current children cumulated CPU time (s) 9.06
Current children cumulated vsize (KiB) 346352

[startup+3.10063 s]
/proc/loadavg: 7.36 7.88 7.95 9/179 22596
/proc/meminfo: memFree=14495352/32950928 swapFree=67111528/67111528
[pid=22582] ppid=22580 vsize=379468 CPUtime=21.84 cores=0-7
/proc/22582/stat : 22582 (plingeling) S 22580 22582 22318 0 -1 4202496 68822 1261 0 0 2176 8 0 0 18 0 9 0 589471256 388575232 27113 33554432000 4194304 4417332 140736021182016 18446744073709551615 218892367845 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/22582/statm: 94867 27113 181 55 0 92753 0
[pid=22582/tid=22589] ppid=22580 vsize=379468 CPUtime=2.68 cores=0-7
/proc/22582/task/22589/stat : 22589 (plingeling) R 22580 22582 22318 0 -1 4202560 8849 1261 0 0 268 0 0 0 22 0 9 0 589471296 388575232 27113 33554432000 4194304 4417332 140736021182016 18446744073709551615 4236324 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22582/tid=22590] ppid=22580 vsize=379468 CPUtime=2.68 cores=0-7
/proc/22582/task/22590/stat : 22590 (plingeling) R 22580 22582 22318 0 -1 4202560 4625 1261 0 0 268 0 0 0 22 0 9 0 589471296 388575232 27113 33554432000 4194304 4417332 140736021182016 18446744073709551615 4336619 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22582/tid=22591] ppid=22580 vsize=379468 CPUtime=2.67 cores=0-7
/proc/22582/task/22591/stat : 22591 (plingeling) R 22580 22582 22318 0 -1 4202560 5772 1261 0 0 267 0 0 0 22 0 9 0 589471296 388575232 27113 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22582/tid=22592] ppid=22580 vsize=379468 CPUtime=2.67 cores=0-7
/proc/22582/task/22592/stat : 22592 (plingeling) R 22580 22582 22318 0 -1 4202560 913 1261 0 0 267 0 0 0 23 0 9 0 589471296 388575232 27113 33554432000 4194304 4417332 140736021182016 18446744073709551615 4245068 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22582/tid=22593] ppid=22580 vsize=379468 CPUtime=2.67 cores=0-7
/proc/22582/task/22593/stat : 22593 (plingeling) R 22580 22582 22318 0 -1 4202560 8459 1261 0 0 266 1 0 0 23 0 9 0 589471296 388575232 27113 33554432000 4194304 4417332 140736021182016 18446744073709551615 4252361 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22582/tid=22594] ppid=22580 vsize=379468 CPUtime=2.66 cores=0-7
/proc/22582/task/22594/stat : 22594 (plingeling) R 22580 22582 22318 0 -1 4202560 6747 1261 0 0 266 0 0 0 23 0 9 0 589471297 388575232 27113 33554432000 4194304 4417332 140736021182016 18446744073709551615 4319442 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22582/tid=22595] ppid=22580 vsize=379468 CPUtime=2.67 cores=0-7
/proc/22582/task/22595/stat : 22595 (plingeling) R 22580 22582 22318 0 -1 4202560 10129 1261 0 0 266 1 0 0 22 0 9 0 589471297 388575232 27113 33554432000 4194304 4417332 140736021182016 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22582/tid=22596] ppid=22580 vsize=379468 CPUtime=2.67 cores=0-7
/proc/22582/task/22596/stat : 22596 (plingeling) R 22580 22582 22318 0 -1 4202560 4605 1261 0 0 267 0 0 0 22 0 9 0 589471297 388575232 27113 33554432000 4194304 4417332 140736021182016 18446744073709551615 4257432 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 21.84
Current children cumulated vsize (KiB) 379468

[startup+6.30707 s]
/proc/loadavg: 7.41 7.88 7.95 9/179 22596
/proc/meminfo: memFree=14474396/32950928 swapFree=67111528/67111528
[pid=22582] ppid=22580 vsize=396544 CPUtime=47.48 cores=0-7
/proc/22582/stat : 22582 (plingeling) S 22580 22582 22318 0 -1 4202496 76458 1261 0 0 4738 10 0 0 18 0 9 0 589471256 406061056 33036 33554432000 4194304 4417332 140736021182016 18446744073709551615 218892367845 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/22582/statm: 99136 33036 181 55 0 97022 0
[pid=22582/tid=22589] ppid=22580 vsize=396544 CPUtime=5.89 cores=0-7
/proc/22582/task/22589/stat : 22589 (plingeling) R 22580 22582 22318 0 -1 4202560 10393 1261 0 0 588 1 0 0 25 0 9 0 589471296 406061056 33036 33554432000 4194304 4417332 140736021182016 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22582/tid=22590] ppid=22580 vsize=396544 CPUtime=5.89 cores=0-7
/proc/22582/task/22590/stat : 22590 (plingeling) R 22580 22582 22318 0 -1 4202560 7122 1261 0 0 588 1 0 0 25 0 9 0 589471296 406061056 33036 33554432000 4194304 4417332 140736021182016 18446744073709551615 4358020 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22582/tid=22591] ppid=22580 vsize=396544 CPUtime=5.88 cores=0-7
/proc/22582/task/22591/stat : 22591 (plingeling) R 22580 22582 22318 0 -1 4202560 6578 1261 0 0 588 0 0 0 25 0 9 0 589471296 406061056 33036 33554432000 4194304 4417332 140736021182016 18446744073709551615 4236995 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22582/tid=22592] ppid=22580 vsize=396544 CPUtime=5.88 cores=0-7
/proc/22582/task/22592/stat : 22592 (plingeling) R 22580 22582 22318 0 -1 4202560 1578 1261 0 0 588 0 0 0 25 0 9 0 589471296 406061056 33036 33554432000 4194304 4417332 140736021182016 18446744073709551615 4365859 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22582/tid=22593] ppid=22580 vsize=396544 CPUtime=5.88 cores=0-7
/proc/22582/task/22593/stat : 22593 (plingeling) R 22580 22582 22318 0 -1 4202560 9051 1261 0 0 587 1 0 0 25 0 9 0 589471296 406061056 33036 33554432000 4194304 4417332 140736021182016 18446744073709551615 4233081 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22582/tid=22594] ppid=22580 vsize=396544 CPUtime=5.85 cores=0-7
/proc/22582/task/22594/stat : 22594 (plingeling) R 22580 22582 22318 0 -1 4202560 7035 1261 0 0 585 0 0 0 25 0 9 0 589471297 406061056 33036 33554432000 4194304 4417332 140736021182016 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22582/tid=22595] ppid=22580 vsize=396544 CPUtime=5.88 cores=0-7
/proc/22582/task/22595/stat : 22595 (plingeling) R 22580 22582 22318 0 -1 4202560 11373 1261 0 0 586 2 0 0 25 0 9 0 589471297 406061056 33036 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364329 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22582/tid=22596] ppid=22580 vsize=396544 CPUtime=5.87 cores=0-7
/proc/22582/task/22596/stat : 22596 (plingeling) R 22580 22582 22318 0 -1 4202560 4605 1261 0 0 587 0 0 0 25 0 9 0 589471297 406061056 33036 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364671 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 47.48
Current children cumulated vsize (KiB) 396544

[startup+12.701 s]
/proc/loadavg: 7.46 7.88 7.95 9/179 22596
/proc/meminfo: memFree=14443040/32950928 swapFree=67111528/67111528
[pid=22582] ppid=22580 vsize=421800 CPUtime=98.59 cores=0-7
/proc/22582/stat : 22582 (plingeling) S 22580 22582 22318 0 -1 4202496 95095 1261 0 0 9847 12 0 0 18 0 9 0 589471256 431923200 38883 33554432000 4194304 4417332 140736021182016 18446744073709551615 218892367845 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/22582/statm: 105450 38883 181 55 0 103336 0
[pid=22582/tid=22589] ppid=22580 vsize=421800 CPUtime=12.28 cores=0-7
/proc/22582/task/22589/stat : 22589 (plingeling) R 22580 22582 22318 0 -1 4202560 13502 1261 0 0 1227 1 0 0 25 0 9 0 589471296 431923200 38883 33554432000 4194304 4417332 140736021182016 18446744073709551615 4353461 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22582/tid=22590] ppid=22580 vsize=421800 CPUtime=12.28 cores=0-7
/proc/22582/task/22590/stat : 22590 (plingeling) R 22580 22582 22318 0 -1 4202560 8864 1261 0 0 1227 1 0 0 25 0 9 0 589471296 431923200 38883 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22582/tid=22591] ppid=22580 vsize=421800 CPUtime=12.28 cores=0-7
/proc/22582/task/22591/stat : 22591 (plingeling) R 22580 22582 22318 0 -1 4202560 7206 1261 0 0 1227 1 0 0 25 0 9 0 589471296 431923200 38883 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364383 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22582/tid=22592] ppid=22580 vsize=421800 CPUtime=12.27 cores=0-7
/proc/22582/task/22592/stat : 22592 (plingeling) R 22580 22582 22318 0 -1 4202560 5025 1261 0 0 1227 0 0 0 25 0 9 0 589471296 431923200 38883 33554432000 4194304 4417332 140736021182016 18446744073709551615 4353424 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22582/tid=22593] ppid=22580 vsize=421800 CPUtime=12.28 cores=0-7
/proc/22582/task/22593/stat : 22593 (plingeling) R 22580 22582 22318 0 -1 4202560 14274 1261 0 0 1226 2 0 0 25 0 9 0 589471296 431923200 38883 33554432000 4194304 4417332 140736021182016 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22582/tid=22594] ppid=22580 vsize=421800 CPUtime=12.21 cores=0-7
/proc/22582/task/22594/stat : 22594 (plingeling) R 22580 22582 22318 0 -1 4202560 9997 1261 0 0 1221 0 0 0 25 0 9 0 589471297 431923200 38883 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22582/tid=22595] ppid=22580 vsize=421800 CPUtime=12.27 cores=0-7
/proc/22582/task/22595/stat : 22595 (plingeling) R 22580 22582 22318 0 -1 4202560 11840 1261 0 0 1225 2 0 0 25 0 9 0 589471297 431923200 38883 33554432000 4194304 4417332 140736021182016 18446744073709551615 4353484 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22582/tid=22596] ppid=22580 vsize=421800 CPUtime=12.26 cores=0-7
/proc/22582/task/22596/stat : 22596 (plingeling) R 22580 22582 22318 0 -1 4202560 5664 1261 0 0 1226 0 0 0 25 0 9 0 589471297 431923200 38883 33554432000 4194304 4417332 140736021182016 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 98.59
Current children cumulated vsize (KiB) 421800

[startup+25.5007 s]
/proc/loadavg: 7.58 7.89 7.95 9/179 22597
/proc/meminfo: memFree=14394780/32950928 swapFree=67111528/67111528
[pid=22582] ppid=22580 vsize=456708 CPUtime=200.92 cores=0-7
/proc/22582/stat : 22582 (plingeling) S 22580 22582 22318 0 -1 4202496 151595 1261 0 0 20070 22 0 0 18 0 9 0 589471256 467668992 49994 33554432000 4194304 4417332 140736021182016 18446744073709551615 218892367845 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/22582/statm: 114177 49994 181 55 0 112063 0
[pid=22582/tid=22589] ppid=22580 vsize=456708 CPUtime=25.08 cores=0-7
/proc/22582/task/22589/stat : 22589 (plingeling) R 22580 22582 22318 0 -1 4202560 20015 1261 0 0 2506 2 0 0 25 0 9 0 589471296 467668992 49994 33554432000 4194304 4417332 140736021182016 18446744073709551615 4250915 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22582/tid=22590] ppid=22580 vsize=456708 CPUtime=25.08 cores=0-7
/proc/22582/task/22590/stat : 22590 (plingeling) R 22580 22582 22318 0 -1 4202560 17337 1261 0 0 2505 3 0 0 25 0 9 0 589471296 467668992 49994 33554432000 4194304 4417332 140736021182016 18446744073709551615 4356735 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22582/tid=22591] ppid=22580 vsize=456708 CPUtime=25.08 cores=0-7
/proc/22582/task/22591/stat : 22591 (plingeling) R 22580 22582 22318 0 -1 4202560 11417 1261 0 0 2507 1 0 0 25 0 9 0 589471296 467668992 49994 33554432000 4194304 4417332 140736021182016 18446744073709551615 4236755 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22582/tid=22592] ppid=22580 vsize=456708 CPUtime=25.06 cores=0-7
/proc/22582/task/22592/stat : 22592 (plingeling) R 22580 22582 22318 0 -1 4202560 17099 1261 0 0 2504 2 0 0 25 0 9 0 589471296 467668992 49994 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22582/tid=22593] ppid=22580 vsize=456708 CPUtime=25.07 cores=0-7
/proc/22582/task/22593/stat : 22593 (plingeling) R 22580 22582 22318 0 -1 4202560 25779 1261 0 0 2504 3 0 0 25 0 9 0 589471296 467668992 49994 33554432000 4194304 4417332 140736021182016 18446744073709551615 4346399 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22582/tid=22594] ppid=22580 vsize=456708 CPUtime=24.94 cores=0-7
/proc/22582/task/22594/stat : 22594 (plingeling) R 22580 22582 22318 0 -1 4202560 16560 1261 0 0 2492 2 0 0 25 0 9 0 589471297 467668992 49994 33554432000 4194304 4417332 140736021182016 18446744073709551615 4357164 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22582/tid=22595] ppid=22580 vsize=456708 CPUtime=25.07 cores=0-7
/proc/22582/task/22595/stat : 22595 (plingeling) R 22580 22582 22318 0 -1 4202560 16604 1261 0 0 2504 3 0 0 25 0 9 0 589471297 467668992 49994 33554432000 4194304 4417332 140736021182016 18446744073709551615 4365846 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22582/tid=22596] ppid=22580 vsize=456708 CPUtime=25.07 cores=0-7
/proc/22582/task/22596/stat : 22596 (plingeling) R 22580 22582 22318 0 -1 4202560 8061 1261 0 0 2506 1 0 0 25 0 9 0 589471297 467668992 49994 33554432000 4194304 4417332 140736021182016 18446744073709551615 4233477 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 200.92
Current children cumulated vsize (KiB) 456708

[startup+51.1012 s]
/proc/loadavg: 7.72 7.89 7.95 9/179 22597
/proc/meminfo: memFree=14339544/32950928 swapFree=67111528/67111528
[pid=22582] ppid=22580 vsize=487972 CPUtime=405.58 cores=0-7
/proc/22582/stat : 22582 (plingeling) S 22580 22582 22318 0 -1 4202496 265800 1261 0 0 40517 41 0 0 18 0 9 0 589471256 499683328 58817 33554432000 4194304 4417332 140736021182016 18446744073709551615 218892367845 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/22582/statm: 121993 58817 181 55 0 119879 0
[pid=22582/tid=22589] ppid=22580 vsize=487972 CPUtime=50.68 cores=0-7
/proc/22582/task/22589/stat : 22589 (plingeling) R 22580 22582 22318 0 -1 4202560 39060 1261 0 0 5063 5 0 0 25 0 9 0 589471296 499683328 58817 33554432000 4194304 4417332 140736021182016 18446744073709551615 4366011 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22582/tid=22590] ppid=22580 vsize=487972 CPUtime=50.68 cores=0-7
/proc/22582/task/22590/stat : 22590 (plingeling) R 22580 22582 22318 0 -1 4202560 23200 1261 0 0 5064 4 0 0 25 0 9 0 589471296 499683328 58817 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22582/tid=22591] ppid=22580 vsize=487972 CPUtime=50.68 cores=0-7
/proc/22582/task/22591/stat : 22591 (plingeling) R 22580 22582 22318 0 -1 4202560 13694 1261 0 0 5067 1 0 0 25 0 9 0 589471296 499683328 58817 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22582/tid=22592] ppid=22580 vsize=487972 CPUtime=50.67 cores=0-7
/proc/22582/task/22592/stat : 22592 (plingeling) R 22580 22582 22318 0 -1 4202560 55557 1261 0 0 5058 9 0 0 25 0 9 0 589471296 499683328 58817 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364362 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22582/tid=22593] ppid=22580 vsize=487972 CPUtime=50.67 cores=0-7
/proc/22582/task/22593/stat : 22593 (plingeling) R 22580 22582 22318 0 -1 4202560 42943 1261 0 0 5060 7 0 0 25 0 9 0 589471296 499683328 58817 33554432000 4194304 4417332 140736021182016 18446744073709551615 4236835 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22582/tid=22594] ppid=22580 vsize=487972 CPUtime=50.41 cores=0-7
/proc/22582/task/22594/stat : 22594 (plingeling) R 22580 22582 22318 0 -1 4202560 32534 1261 0 0 5036 5 0 0 25 0 9 0 589471297 499683328 58817 33554432000 4194304 4417332 140736021182016 18446744073709551615 4237169 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22582/tid=22595] ppid=22580 vsize=487972 CPUtime=50.67 cores=0-7
/proc/22582/task/22595/stat : 22595 (plingeling) R 22580 22582 22318 0 -1 4202560 26634 1261 0 0 5063 4 0 0 25 0 9 0 589471297 499683328 58817 33554432000 4194304 4417332 140736021182016 18446744073709551615 4356907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22582/tid=22596] ppid=22580 vsize=487972 CPUtime=50.66 cores=0-7
/proc/22582/task/22596/stat : 22596 (plingeling) R 22580 22582 22318 0 -1 4202560 13455 1261 0 0 5065 1 0 0 25 0 9 0 589471297 499683328 58817 33554432000 4194304 4417332 140736021182016 18446744073709551615 4366485 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 405.58
Current children cumulated vsize (KiB) 487972

[startup+102.301 s]
/proc/loadavg: 7.88 7.91 7.95 9/179 22599

################
# More data... #
################

/proc/22582/task/22592/stat : 22592 (plingeling) R 22580 22582 22318 0 -1 4202560 973372 1261 0 0 471953 197 0 0 25 0 9 0 589471296 1930297344 406261 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22582/tid=22593] ppid=22580 vsize=1885056 CPUtime=4721.96 cores=0-7
/proc/22582/task/22593/stat : 22593 (plingeling) R 22580 22582 22318 0 -1 4202560 1371002 1261 0 0 471964 232 0 0 25 0 9 0 589471296 1930297344 406261 33554432000 4194304 4417332 140736021182016 18446744073709551615 4366297 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22582/tid=22594] ppid=22580 vsize=1885056 CPUtime=4719.79 cores=0-7
/proc/22582/task/22594/stat : 22594 (plingeling) R 22580 22582 22318 0 -1 4202560 1564764 1261 0 0 471688 291 0 0 25 0 9 0 589471297 1930297344 406261 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22582/tid=22595] ppid=22580 vsize=1885056 CPUtime=4697.38 cores=0-7
/proc/22582/task/22595/stat : 22595 (plingeling) R 22580 22582 22318 0 -1 4202560 1227199 1261 0 0 469509 229 0 0 25 0 9 0 589471297 1930297344 406261 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364547 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22582/tid=22596] ppid=22580 vsize=1885056 CPUtime=4721.93 cores=0-7
/proc/22582/task/22596/stat : 22596 (plingeling) R 22580 22582 22318 0 -1 4202560 1021477 1261 0 0 472024 169 0 0 25 0 9 0 589471297 1930297344 406261 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37748.8
Current children cumulated vsize (KiB) 1885056

[startup+4782.3 s]
/proc/loadavg: 8.08 8.06 8.00 9/179 22736
/proc/meminfo: memFree=12502480/32950928 swapFree=67111528/67111528
[pid=22582] ppid=22580 vsize=2361996 CPUtime=38228.5 cores=0-7
/proc/22582/stat : 22582 (plingeling) S 22580 22582 22318 0 -1 4202496 10906326 1261 0 0 3820832 2018 0 0 18 0 9 0 589471256 2418683904 525496 33554432000 4194304 4417332 140736021182016 18446744073709551615 218892367845 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/22582/statm: 590499 525496 181 55 0 588385 0
[pid=22582/tid=22589] ppid=22580 vsize=2361996 CPUtime=4781.93 cores=0-7
/proc/22582/task/22589/stat : 22589 (plingeling) R 22580 22582 22318 0 -1 4202560 1857303 1261 0 0 477856 337 0 0 25 0 9 0 589471296 2418683904 525496 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364713 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22582/tid=22590] ppid=22580 vsize=2361996 CPUtime=4781.94 cores=0-7
/proc/22582/task/22590/stat : 22590 (plingeling) R 22580 22582 22318 0 -1 4202560 1400613 1261 0 0 477930 264 0 0 25 0 9 0 589471296 2418683904 525496 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22582/tid=22591] ppid=22580 vsize=2361996 CPUtime=4781.94 cores=0-7
/proc/22582/task/22591/stat : 22591 (plingeling) R 22580 22582 22318 0 -1 4202560 1384766 1261 0 0 477921 273 0 0 25 0 9 0 589471296 2418683904 525496 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22582/tid=22592] ppid=22580 vsize=2361996 CPUtime=4781.48 cores=0-7
/proc/22582/task/22592/stat : 22592 (plingeling) R 22580 22582 22318 0 -1 4202560 982007 1261 0 0 477949 199 0 0 25 0 9 0 589471296 2418683904 525496 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22582/tid=22593] ppid=22580 vsize=2361996 CPUtime=4781.96 cores=0-7
/proc/22582/task/22593/stat : 22593 (plingeling) R 22580 22582 22318 0 -1 4202560 1371002 1261 0 0 477964 232 0 0 25 0 9 0 589471296 2418683904 525496 33554432000 4194304 4417332 140736021182016 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22582/tid=22594] ppid=22580 vsize=2361996 CPUtime=4779.78 cores=0-7
/proc/22582/task/22594/stat : 22594 (plingeling) R 22580 22582 22318 0 -1 4202560 1597532 1261 0 0 477677 301 0 0 25 0 9 0 589471297 2418683904 525496 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22582/tid=22595] ppid=22580 vsize=2361996 CPUtime=4757.05 cores=0-7
/proc/22582/task/22595/stat : 22595 (plingeling) R 22580 22582 22318 0 -1 4202560 1254868 1261 0 0 475473 232 0 0 25 0 9 0 589471297 2418683904 525496 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22582/tid=22596] ppid=22580 vsize=2361996 CPUtime=4781.93 cores=0-7
/proc/22582/task/22596/stat : 22596 (plingeling) R 22580 22582 22318 0 -1 4202560 1039512 1261 0 0 478020 173 0 0 25 0 9 0 589471297 2418683904 525496 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38228.5
Current children cumulated vsize (KiB) 2361996

[startup+4842.3 s]
/proc/loadavg: 8.08 8.06 8.00 9/179 22737
/proc/meminfo: memFree=13039112/32950928 swapFree=67111528/67111528
[pid=22582] ppid=22580 vsize=1826208 CPUtime=38708.2 cores=0-7
/proc/22582/stat : 22582 (plingeling) S 22580 22582 22318 0 -1 4202496 10993863 1261 0 0 3868784 2034 0 0 18 0 9 0 589471256 1870036992 391549 33554432000 4194304 4417332 140736021182016 18446744073709551615 218892367845 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/22582/statm: 456552 391549 181 55 0 454438 0
[pid=22582/tid=22589] ppid=22580 vsize=1826208 CPUtime=4841.93 cores=0-7
/proc/22582/task/22589/stat : 22589 (plingeling) R 22580 22582 22318 0 -1 4202560 1901463 1261 0 0 483848 345 0 0 25 0 9 0 589471296 1870036992 391549 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364957 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22582/tid=22590] ppid=22580 vsize=1826208 CPUtime=4841.94 cores=0-7
/proc/22582/task/22590/stat : 22590 (plingeling) R 22580 22582 22318 0 -1 4202560 1412812 1261 0 0 483928 266 0 0 25 0 9 0 589471296 1870036992 391549 33554432000 4194304 4417332 140736021182016 18446744073709551615 4236812 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22582/tid=22591] ppid=22580 vsize=1826208 CPUtime=4841.94 cores=0-7
/proc/22582/task/22591/stat : 22591 (plingeling) R 22580 22582 22318 0 -1 4202560 1384766 1261 0 0 483921 273 0 0 25 0 9 0 589471296 1870036992 391549 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22582/tid=22592] ppid=22580 vsize=1826208 CPUtime=4841.48 cores=0-7
/proc/22582/task/22592/stat : 22592 (plingeling) R 22580 22582 22318 0 -1 4202560 982007 1261 0 0 483949 199 0 0 25 0 9 0 589471296 1870036992 391549 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22582/tid=22593] ppid=22580 vsize=1826208 CPUtime=4841.95 cores=0-7
/proc/22582/task/22593/stat : 22593 (plingeling) R 22580 22582 22318 0 -1 4202560 1387387 1261 0 0 483962 233 0 0 25 0 9 0 589471296 1870036992 391549 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22582/tid=22594] ppid=22580 vsize=1826208 CPUtime=4839.79 cores=0-7
/proc/22582/task/22594/stat : 22594 (plingeling) R 22580 22582 22318 0 -1 4202560 1599404 1261 0 0 483676 303 0 0 25 0 9 0 589471297 1870036992 391549 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22582/tid=22595] ppid=22580 vsize=1826208 CPUtime=4816.73 cores=0-7
/proc/22582/task/22595/stat : 22595 (plingeling) R 22580 22582 22318 0 -1 4202560 1256061 1261 0 0 481440 233 0 0 25 0 9 0 589471297 1870036992 391549 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22582/tid=22596] ppid=22580 vsize=1826208 CPUtime=4841.93 cores=0-7
/proc/22582/task/22596/stat : 22596 (plingeling) R 22580 22582 22318 0 -1 4202560 1051240 1261 0 0 484018 175 0 0 25 0 9 0 589471297 1870036992 391549 33554432000 4194304 4417332 140736021182016 18446744073709551615 4365018 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38708.2
Current children cumulated vsize (KiB) 1826208

[startup+4902.3 s]
/proc/loadavg: 8.07 8.06 8.00 9/179 22739
/proc/meminfo: memFree=12750888/32950928 swapFree=67111528/67111528
[pid=22582] ppid=22580 vsize=2113740 CPUtime=39187.9 cores=0-7
/proc/22582/stat : 22582 (plingeling) S 22580 22582 22318 0 -1 4202496 11160498 1261 0 0 3916717 2072 0 0 18 0 9 0 589471256 2164469760 463432 33554432000 4194304 4417332 140736021182016 18446744073709551615 218892367845 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/22582/statm: 528435 463432 181 55 0 526321 0
[pid=22582/tid=22589] ppid=22580 vsize=2113740 CPUtime=4901.94 cores=0-7
/proc/22582/task/22589/stat : 22589 (plingeling) R 22580 22582 22318 0 -1 4202560 1936860 1261 0 0 489843 351 0 0 25 0 9 0 589471296 2164469760 463432 33554432000 4194304 4417332 140736021182016 18446744073709551615 4264950 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22582/tid=22590] ppid=22580 vsize=2113740 CPUtime=4901.95 cores=0-7
/proc/22582/task/22590/stat : 22590 (plingeling) R 22580 22582 22318 0 -1 4202560 1431048 1261 0 0 489926 269 0 0 25 0 9 0 589471296 2164469760 463432 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22582/tid=22591] ppid=22580 vsize=2113740 CPUtime=4901.95 cores=0-7
/proc/22582/task/22591/stat : 22591 (plingeling) R 22580 22582 22318 0 -1 4202560 1454918 1261 0 0 489900 295 0 0 25 0 9 0 589471296 2164469760 463432 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22582/tid=22592] ppid=22580 vsize=2113740 CPUtime=4901.49 cores=0-7
/proc/22582/task/22592/stat : 22592 (plingeling) R 22580 22582 22318 0 -1 4202560 988760 1261 0 0 489948 201 0 0 25 0 9 0 589471296 2164469760 463432 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22582/tid=22593] ppid=22580 vsize=2113740 CPUtime=4901.96 cores=0-7
/proc/22582/task/22593/stat : 22593 (plingeling) R 22580 22582 22318 0 -1 4202560 1387387 1261 0 0 489963 233 0 0 25 0 9 0 589471296 2164469760 463432 33554432000 4194304 4417332 140736021182016 18446744073709551615 18446744073699065911 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22582/tid=22594] ppid=22580 vsize=2113740 CPUtime=4899.79 cores=0-7
/proc/22582/task/22594/stat : 22594 (plingeling) R 22580 22582 22318 0 -1 4202560 1614488 1261 0 0 489673 306 0 0 25 0 9 0 589471297 2164469760 463432 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22582/tid=22595] ppid=22580 vsize=2113740 CPUtime=4876.4 cores=0-7
/proc/22582/task/22595/stat : 22595 (plingeling) R 22580 22582 22318 0 -1 4202560 1264444 1261 0 0 487406 234 0 0 25 0 9 0 589471297 2164469760 463432 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22582/tid=22596] ppid=22580 vsize=2113740 CPUtime=4901.95 cores=0-7
/proc/22582/task/22596/stat : 22596 (plingeling) R 22580 22582 22318 0 -1 4202560 1063870 1261 0 0 490018 177 0 0 25 0 9 0 589471297 2164469760 463432 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39187.9
Current children cumulated vsize (KiB) 2113740

[startup+4962.3 s]
/proc/loadavg: 8.02 8.05 8.00 9/179 22740
/proc/meminfo: memFree=12939320/32950928 swapFree=67111528/67111528
[pid=22582] ppid=22580 vsize=1925612 CPUtime=39667.5 cores=0-7
/proc/22582/stat : 22582 (plingeling) S 22580 22582 22318 0 -1 4202496 11302726 1261 0 0 3964651 2102 0 0 18 0 9 0 589471256 1971826688 416400 33554432000 4194304 4417332 140736021182016 18446744073709551615 218892367845 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/22582/statm: 481403 416400 181 55 0 479289 0
[pid=22582/tid=22589] ppid=22580 vsize=1925612 CPUtime=4961.93 cores=0-7
/proc/22582/task/22589/stat : 22589 (plingeling) R 22580 22582 22318 0 -1 4202560 1968114 1261 0 0 495835 358 0 0 25 0 9 0 589471296 1971826688 416400 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22582/tid=22590] ppid=22580 vsize=1925612 CPUtime=4961.94 cores=0-7
/proc/22582/task/22590/stat : 22590 (plingeling) R 22580 22582 22318 0 -1 4202560 1449155 1261 0 0 495923 271 0 0 25 0 9 0 589471296 1971826688 416400 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22582/tid=22591] ppid=22580 vsize=1925612 CPUtime=4961.95 cores=0-7
/proc/22582/task/22591/stat : 22591 (plingeling) R 22580 22582 22318 0 -1 4202560 1507831 1261 0 0 495886 309 0 0 25 0 9 0 589471296 1971826688 416400 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22582/tid=22592] ppid=22580 vsize=1925612 CPUtime=4961.48 cores=0-7
/proc/22582/task/22592/stat : 22592 (plingeling) R 22580 22582 22318 0 -1 4202560 996446 1261 0 0 495946 202 0 0 25 0 9 0 589471296 1971826688 416400 33554432000 4194304 4417332 140736021182016 18446744073709551615 4357280 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22582/tid=22593] ppid=22580 vsize=1925612 CPUtime=4961.96 cores=0-7
/proc/22582/task/22593/stat : 22593 (plingeling) R 22580 22582 22318 0 -1 4202560 1387387 1261 0 0 495963 233 0 0 25 0 9 0 589471296 1971826688 416400 33554432000 4194304 4417332 140736021182016 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22582/tid=22594] ppid=22580 vsize=1925612 CPUtime=4959.78 cores=0-7
/proc/22582/task/22594/stat : 22594 (plingeling) R 22580 22582 22318 0 -1 4202560 1627496 1261 0 0 495668 310 0 0 25 0 9 0 589471297 1971826688 416400 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22582/tid=22595] ppid=22580 vsize=1925612 CPUtime=4936.07 cores=0-7
/proc/22582/task/22595/stat : 22595 (plingeling) R 22580 22582 22318 0 -1 4202560 1274024 1261 0 0 493372 235 0 0 25 0 9 0 589471297 1971826688 416400 33554432000 4194304 4417332 140736021182016 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22582/tid=22596] ppid=22580 vsize=1925612 CPUtime=4961.94 cores=0-7
/proc/22582/task/22596/stat : 22596 (plingeling) R 22580 22582 22318 0 -1 4202560 1073550 1261 0 0 496016 178 0 0 25 0 9 0 589471297 1971826688 416400 33554432000 4194304 4417332 140736021182016 18446744073709551615 4366038 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39667.5
Current children cumulated vsize (KiB) 1925612



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5003.9 s]
/proc/loadavg: 8.01 8.04 8.00 9/179 22741
/proc/meminfo: memFree=12926644/32950928 swapFree=67111528/67111528
[pid=22582] ppid=22580 vsize=1938160 CPUtime=40000.1 cores=0-7
/proc/22582/stat : 22582 (plingeling) S 22580 22582 22318 0 -1 4202496 11368516 1261 0 0 3997900 2115 0 0 18 0 9 0 589471256 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 218892367845 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/22582/statm: 484540 419537 181 55 0 482426 0
[pid=22582/tid=22589] ppid=22580 vsize=1938160 CPUtime=5003.54 cores=0-7
/proc/22582/task/22589/stat : 22589 (plingeling) R 22580 22582 22318 0 -1 4202560 1985943 1261 0 0 499992 362 0 0 25 0 9 0 589471296 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22582/tid=22590] ppid=22580 vsize=1938160 CPUtime=5003.56 cores=0-7
/proc/22582/task/22590/stat : 22590 (plingeling) R 22580 22582 22318 0 -1 4202560 1449155 1261 0 0 500084 272 0 0 25 0 9 0 589471296 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364942 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22582/tid=22591] ppid=22580 vsize=1938160 CPUtime=5003.56 cores=0-7
/proc/22582/task/22591/stat : 22591 (plingeling) R 22580 22582 22318 0 -1 4202560 1507831 1261 0 0 500047 309 0 0 25 0 9 0 589471296 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22582/tid=22592] ppid=22580 vsize=1938160 CPUtime=5003.07 cores=0-7
/proc/22582/task/22592/stat : 22592 (plingeling) R 22580 22582 22318 0 -1 4202560 1011270 1261 0 0 500104 203 0 0 25 0 9 0 589471296 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 4236995 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22582/tid=22593] ppid=22580 vsize=1938160 CPUtime=5003.57 cores=0-7
/proc/22582/task/22593/stat : 22593 (plingeling) R 22580 22582 22318 0 -1 4202560 1401916 1261 0 0 500122 235 0 0 25 0 9 0 589471296 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22582/tid=22594] ppid=22580 vsize=1938160 CPUtime=5001.39 cores=0-7
/proc/22582/task/22594/stat : 22594 (plingeling) R 22580 22582 22318 0 -1 4202560 1631434 1261 0 0 499828 311 0 0 25 0 9 0 589471297 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 4354414 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22582/tid=22595] ppid=22580 vsize=1938160 CPUtime=4977.45 cores=0-7
/proc/22582/task/22595/stat : 22595 (plingeling) R 22580 22582 22318 0 -1 4202560 1276366 1261 0 0 497509 236 0 0 25 0 9 0 589471297 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22582/tid=22596] ppid=22580 vsize=1938160 CPUtime=5003.54 cores=0-7
/proc/22582/task/22596/stat : 22596 (plingeling) R 22580 22582 22318 0 -1 4202560 1085878 1261 0 0 500173 181 0 0 25 0 9 0 589471297 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 1938160

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5003.9 s]
/proc/loadavg: 8.01 8.04 8.00 9/179 22741
/proc/meminfo: memFree=12926644/32950928 swapFree=67111528/67111528
[pid=22582] ppid=22580 vsize=1938160 CPUtime=40000.1 cores=0-7
/proc/22582/stat : 22582 (plingeling) S 22580 22582 22318 0 -1 4202496 11368516 1261 0 0 3997900 2115 0 0 18 0 9 0 589471256 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 218892367845 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/22582/statm: 484540 419537 181 55 0 482426 0
[pid=22582/tid=22589] ppid=22580 vsize=1938160 CPUtime=5003.54 cores=0-7
/proc/22582/task/22589/stat : 22589 (plingeling) R 22580 22582 22318 0 -1 4202560 1985943 1261 0 0 499992 362 0 0 25 0 9 0 589471296 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22582/tid=22590] ppid=22580 vsize=1938160 CPUtime=5003.56 cores=0-7
/proc/22582/task/22590/stat : 22590 (plingeling) R 22580 22582 22318 0 -1 4202560 1449155 1261 0 0 500084 272 0 0 25 0 9 0 589471296 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364942 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22582/tid=22591] ppid=22580 vsize=1938160 CPUtime=5003.56 cores=0-7
/proc/22582/task/22591/stat : 22591 (plingeling) R 22580 22582 22318 0 -1 4202560 1507831 1261 0 0 500047 309 0 0 25 0 9 0 589471296 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22582/tid=22592] ppid=22580 vsize=1938160 CPUtime=5003.07 cores=0-7
/proc/22582/task/22592/stat : 22592 (plingeling) R 22580 22582 22318 0 -1 4202560 1011270 1261 0 0 500104 203 0 0 25 0 9 0 589471296 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 4236995 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22582/tid=22593] ppid=22580 vsize=1938160 CPUtime=5003.57 cores=0-7
/proc/22582/task/22593/stat : 22593 (plingeling) R 22580 22582 22318 0 -1 4202560 1401916 1261 0 0 500122 235 0 0 25 0 9 0 589471296 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22582/tid=22594] ppid=22580 vsize=1938160 CPUtime=5001.39 cores=0-7
/proc/22582/task/22594/stat : 22594 (plingeling) R 22580 22582 22318 0 -1 4202560 1631434 1261 0 0 499828 311 0 0 25 0 9 0 589471297 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 4354414 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22582/tid=22595] ppid=22580 vsize=1938160 CPUtime=4977.45 cores=0-7
/proc/22582/task/22595/stat : 22595 (plingeling) R 22580 22582 22318 0 -1 4202560 1276366 1261 0 0 497509 236 0 0 25 0 9 0 589471297 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22582/tid=22596] ppid=22580 vsize=1938160 CPUtime=5003.54 cores=0-7
/proc/22582/task/22596/stat : 22596 (plingeling) R 22580 22582 22318 0 -1 4202560 1085878 1261 0 0 500173 181 0 0 25 0 9 0 589471297 1984675840 419537 33554432000 4194304 4417332 140736021182016 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 1938160

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5004
CPU time (s): 40000.3
CPU user time (s): 39979
CPU system time (s): 21.2448
CPU usage (%): 799.366
Max. virtual memory (cumulated for all children) (KiB): 2500620

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39979
system time used= 21.2448
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 11369777
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 745
involuntary context switches= 191930

runsolver used 7.77182 second user time and 19.519 second system time

The end

Launcher Data 

Begin job on node112 at 2011-04-28 14:06:46
IDJOB=3309282
IDBENCH=20523
IDSOLVER=1695
FILE ID=node112/3309282-1303992406
RUNJOBID= node112-1303987384-22332
PBS_JOBID= 13168762
Free space on /tmp= 72564 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT07/industrial/IBM_FV_2002_13_rule_1/SAT_dat.k45.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3309282-1303992406/watcher-3309282-1303992406 -o /tmp/evaluation-result-3309282-1303992406/solver-3309282-1303992406 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3309282-1303992406.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 8a76c8add64173ae39149aae572a02d9
RANDOM SEED=1008793358

node112.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5261.27
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      14604072 kB
Buffers:       1991944 kB
Cached:       15732284 kB
SwapCached:          0 kB
Active:        6779268 kB
Inactive:     11009924 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      14604072 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            3916 kB
Writeback:           0 kB
AnonPages:       65084 kB
Mapped:          14824 kB
Slab:           491428 kB
PageTables:       4352 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   185024 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72560 MiB
End job on node112 at 2011-04-28 15:30:12