Trace number 3372113

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.4 5008.75

General information on the benchmark

Nameindustrial/anbulagan/
hard-unsat/partial-5-11-u.cnf
MD5SUM8e22d84b1ae625d44fbf300a32374a2d
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables164158
Number of clauses730719
Sum of the clauses size1660140
Maximum clause length10
Minimum clause length1
Number of clauses of size 112474
Number of clauses of size 2600009
Number of clauses of size 387256
Number of clauses of size 43440
Number of clauses of size 513760
Number of clauses of size over 513780

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372113-1305435404/watcher-3372113-1305435404 -o /tmp/evaluation-result-3372113-1305435404/solver-3372113-1305435404 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3372113-1305435404.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.94 7.88 7.74 1/340 24854
/proc/meminfo: memFree=25759588/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=13512 CPUtime=0 cores=0-7
/proc/24854/stat : 24854 (plingeling) R 24852 24854 24828 0 -1 4202496 1454 1253 0 0 0 0 0 0 21 0 1 0 655140888 13836288 1384 33554432000 4194304 4417452 140736700049632 18446744073709551615 230808859595 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/24854/statm: 3378 1384 147 55 0 1264 0

[startup+0.0484519 s]
/proc/loadavg: 6.94 7.88 7.74 1/340 24854
/proc/meminfo: memFree=25759588/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=31820 CPUtime=0.03 cores=0-7
/proc/24854/stat : 24854 (plingeling) R 24852 24854 24828 0 -1 4202496 6050 1253 0 0 3 0 0 0 21 0 1 0 655140888 32583680 5978 33554432000 4194304 4417452 140736700049632 18446744073709551615 4260021 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/24854/statm: 7955 5978 148 55 0 5841 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 31820

[startup+0.100441 s]
/proc/loadavg: 6.94 7.88 7.74 1/340 24854
/proc/meminfo: memFree=25759588/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=39516 CPUtime=0.09 cores=0-7
/proc/24854/stat : 24854 (plingeling) R 24852 24854 24828 0 -1 4202496 7974 1253 0 0 8 1 0 0 21 0 1 0 655140888 40464384 7902 33554432000 4194304 4417452 140736700049632 18446744073709551615 4262052 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/24854/statm: 9879 7902 148 55 0 7765 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 39516

[startup+0.300404 s]
/proc/loadavg: 6.94 7.88 7.74 1/340 24854
/proc/meminfo: memFree=25759588/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=104324 CPUtime=0.29 cores=0-7
/proc/24854/stat : 24854 (plingeling) R 24852 24854 24828 0 -1 4202496 24176 1253 0 0 26 3 0 0 22 0 1 0 655140888 106827776 24104 33554432000 4194304 4417452 140736700049632 18446744073709551615 4228096 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/24854/statm: 26081 24104 148 55 0 23967 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 104324

[startup+0.700334 s]
/proc/loadavg: 6.94 7.88 7.74 1/340 24854
/proc/meminfo: memFree=25759588/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=194756 CPUtime=0.69 cores=0-7
/proc/24854/stat : 24854 (plingeling) R 24852 24854 24828 0 -1 4202496 46800 1253 0 0 63 6 0 0 23 0 1 0 655140888 199430144 46712 33554432000 4194304 4417452 140736700049632 18446744073709551615 4231547 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/24854/statm: 48689 46712 148 55 0 46575 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 194756

[startup+1.50019 s]
/proc/loadavg: 6.46 7.76 7.71 2/341 24860
/proc/meminfo: memFree=25532296/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=301284 CPUtime=1.49 cores=0-7
/proc/24854/stat : 24854 (plingeling) R 24852 24854 24828 0 -1 4202496 73433 1253 0 0 140 9 0 0 25 0 1 0 655140888 308514816 73345 33554432000 4194304 4417452 140736700049632 18446744073709551615 4262083 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/24854/statm: 75321 73345 149 55 0 73207 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 301284

[startup+3.10086 s]
/proc/loadavg: 6.46 7.76 7.71 9/349 24869
/proc/meminfo: memFree=25466388/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=437676 CPUtime=12.87 cores=0-7
/proc/24854/stat : 24854 (plingeling) S 24852 24854 24828 0 -1 4202496 127488 1253 0 0 1270 17 0 0 25 0 9 0 655140888 448180224 77829 33554432000 4194304 4417452 140736700049632 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/24854/statm: 109419 77829 175 55 0 107305 0
[pid=24854/tid=24861] ppid=24852 vsize=437676 CPUtime=1.4 cores=0-7
/proc/24854/task/24861/stat : 24861 (plingeling) R 24852 24854 24828 0 -1 4202560 12434 1253 0 0 139 1 0 0 19 0 9 0 655141057 448180224 77829 33554432000 4194304 4417452 140736700049632 18446744073709551615 4287492 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24854/tid=24862] ppid=24852 vsize=437676 CPUtime=1.4 cores=0-7
/proc/24854/task/24862/stat : 24862 (plingeling) R 24852 24854 24828 0 -1 4202560 8172 1253 0 0 139 1 0 0 19 0 9 0 655141057 448180224 77829 33554432000 4194304 4417452 140736700049632 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=24854/tid=24863] ppid=24852 vsize=437676 CPUtime=1.38 cores=0-7
/proc/24854/task/24863/stat : 24863 (plingeling) R 24852 24854 24828 0 -1 4202560 3782 1253 0 0 138 0 0 0 25 0 9 0 655141057 448180224 77829 33554432000 4194304 4417452 140736700049632 18446744073709551615 4287111 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=24854/tid=24864] ppid=24852 vsize=437676 CPUtime=1.37 cores=0-7
/proc/24854/task/24864/stat : 24864 (plingeling) R 24852 24854 24828 0 -1 4202560 2867 1253 0 0 137 0 0 0 19 0 9 0 655141057 448180224 77829 33554432000 4194304 4417452 140736700049632 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24854/tid=24865] ppid=24852 vsize=437676 CPUtime=1.39 cores=0-7
/proc/24854/task/24865/stat : 24865 (plingeling) R 24852 24854 24828 0 -1 4202560 6967 1253 0 0 138 1 0 0 19 0 9 0 655141057 448180224 77829 33554432000 4194304 4417452 140736700049632 18446744073709551615 4286323 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=24854/tid=24866] ppid=24852 vsize=437676 CPUtime=1.4 cores=0-7
/proc/24854/task/24866/stat : 24866 (plingeling) R 24852 24854 24828 0 -1 4202560 6998 1253 0 0 139 1 0 0 19 0 9 0 655141057 448180224 77829 33554432000 4194304 4417452 140736700049632 18446744073709551615 4286414 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=24854/tid=24867] ppid=24852 vsize=437676 CPUtime=1.38 cores=0-7
/proc/24854/task/24867/stat : 24867 (plingeling) R 24852 24854 24828 0 -1 4202560 6410 1253 0 0 138 0 0 0 25 0 9 0 655141057 448180224 77829 33554432000 4194304 4417452 140736700049632 18446744073709551615 4287492 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=24854/tid=24868] ppid=24852 vsize=437676 CPUtime=1.39 cores=0-7
/proc/24854/task/24868/stat : 24868 (plingeling) R 24852 24854 24828 0 -1 4202560 6409 1253 0 0 138 1 0 0 19 0 9 0 655141057 448180224 77829 33554432000 4194304 4417452 140736700049632 18446744073709551615 4286467 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 12.87
Current children cumulated vsize (KiB) 437676

[startup+6.30127 s]
/proc/loadavg: 6.46 7.76 7.71 9/349 24869
/proc/meminfo: memFree=25470624/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=423864 CPUtime=38.43 cores=0-7
/proc/24854/stat : 24854 (plingeling) S 24852 24854 24828 0 -1 4202496 135889 1253 0 0 3824 19 0 0 25 0 9 0 655140888 434036736 69619 33554432000 4194304 4417452 140736700049632 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/24854/statm: 105966 69619 180 55 0 103852 0
[pid=24854/tid=24861] ppid=24852 vsize=423864 CPUtime=4.59 cores=0-7
/proc/24854/task/24861/stat : 24861 (plingeling) R 24852 24854 24828 0 -1 4202560 15761 1253 0 0 458 1 0 0 25 0 9 0 655141057 434036736 69619 33554432000 4194304 4417452 140736700049632 18446744073709551615 4271639 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24854/tid=24862] ppid=24852 vsize=423864 CPUtime=4.6 cores=0-7
/proc/24854/task/24862/stat : 24862 (plingeling) R 24852 24854 24828 0 -1 4202560 8861 1253 0 0 459 1 0 0 25 0 9 0 655141057 434036736 69619 33554432000 4194304 4417452 140736700049632 18446744073709551615 4257072 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=24854/tid=24863] ppid=24852 vsize=423864 CPUtime=4.58 cores=0-7
/proc/24854/task/24863/stat : 24863 (plingeling) R 24852 24854 24828 0 -1 4202560 4102 1253 0 0 458 0 0 0 25 0 9 0 655141057 434036736 69619 33554432000 4194304 4417452 140736700049632 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=24854/tid=24864] ppid=24852 vsize=423864 CPUtime=4.54 cores=0-7
/proc/24854/task/24864/stat : 24864 (plingeling) R 24852 24854 24828 0 -1 4202560 2867 1253 0 0 454 0 0 0 25 0 9 0 655141057 434036736 69619 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24854/tid=24865] ppid=24852 vsize=423864 CPUtime=4.59 cores=0-7
/proc/24854/task/24865/stat : 24865 (plingeling) R 24852 24854 24828 0 -1 4202560 7719 1253 0 0 458 1 0 0 25 0 9 0 655141057 434036736 69619 33554432000 4194304 4417452 140736700049632 18446744073709551615 4252087 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=24854/tid=24866] ppid=24852 vsize=423864 CPUtime=4.6 cores=0-7
/proc/24854/task/24866/stat : 24866 (plingeling) R 24852 24854 24828 0 -1 4202560 7747 1253 0 0 459 1 0 0 25 0 9 0 655141057 434036736 69619 33554432000 4194304 4417452 140736700049632 18446744073709551615 4252113 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=24854/tid=24867] ppid=24852 vsize=423864 CPUtime=4.59 cores=0-7
/proc/24854/task/24867/stat : 24867 (plingeling) R 24852 24854 24828 0 -1 4202560 8283 1253 0 0 458 1 0 0 25 0 9 0 655141057 434036736 69619 33554432000 4194304 4417452 140736700049632 18446744073709551615 4251008 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=24854/tid=24868] ppid=24852 vsize=423864 CPUtime=4.59 cores=0-7
/proc/24854/task/24868/stat : 24868 (plingeling) R 24852 24854 24828 0 -1 4202560 7100 1253 0 0 458 1 0 0 25 0 9 0 655141057 434036736 69619 33554432000 4194304 4417452 140736700049632 18446744073709551615 4320455 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38.43
Current children cumulated vsize (KiB) 423864

[startup+12.7011 s]
/proc/loadavg: 6.70 7.77 7.71 9/349 24869
/proc/meminfo: memFree=25519852/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=382288 CPUtime=89.55 cores=0-7
/proc/24854/stat : 24854 (plingeling) S 24852 24854 24828 0 -1 4202496 135889 1253 0 0 8936 19 0 0 25 0 9 0 655140888 391462912 59225 33554432000 4194304 4417452 140736700049632 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/24854/statm: 95572 59225 180 55 0 93458 0
[pid=24854/tid=24861] ppid=24852 vsize=382288 CPUtime=10.99 cores=0-7
/proc/24854/task/24861/stat : 24861 (plingeling) R 24852 24854 24828 0 -1 4202560 15761 1253 0 0 1098 1 0 0 25 0 9 0 655141057 391462912 59225 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364480 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24854/tid=24862] ppid=24852 vsize=382288 CPUtime=11 cores=0-7
/proc/24854/task/24862/stat : 24862 (plingeling) R 24852 24854 24828 0 -1 4202560 8861 1253 0 0 1099 1 0 0 25 0 9 0 655141057 391462912 59225 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=24854/tid=24863] ppid=24852 vsize=382288 CPUtime=10.98 cores=0-7
/proc/24854/task/24863/stat : 24863 (plingeling) R 24852 24854 24828 0 -1 4202560 4102 1253 0 0 1098 0 0 0 25 0 9 0 655141057 391462912 59225 33554432000 4194304 4417452 140736700049632 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=24854/tid=24864] ppid=24852 vsize=382288 CPUtime=10.86 cores=0-7
/proc/24854/task/24864/stat : 24864 (plingeling) R 24852 24854 24828 0 -1 4202560 2867 1253 0 0 1086 0 0 0 25 0 9 0 655141057 391462912 59225 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24854/tid=24865] ppid=24852 vsize=382288 CPUtime=10.99 cores=0-7
/proc/24854/task/24865/stat : 24865 (plingeling) R 24852 24854 24828 0 -1 4202560 7719 1253 0 0 1098 1 0 0 25 0 9 0 655141057 391462912 59225 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364616 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=24854/tid=24866] ppid=24852 vsize=382288 CPUtime=11 cores=0-7
/proc/24854/task/24866/stat : 24866 (plingeling) R 24852 24854 24828 0 -1 4202560 7747 1253 0 0 1099 1 0 0 25 0 9 0 655141057 391462912 59225 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=24854/tid=24867] ppid=24852 vsize=382288 CPUtime=10.98 cores=0-7
/proc/24854/task/24867/stat : 24867 (plingeling) R 24852 24854 24828 0 -1 4202560 8283 1253 0 0 1097 1 0 0 25 0 9 0 655141057 391462912 59225 33554432000 4194304 4417452 140736700049632 18446744073709551615 4237002 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=24854/tid=24868] ppid=24852 vsize=382288 CPUtime=10.99 cores=0-7
/proc/24854/task/24868/stat : 24868 (plingeling) R 24852 24854 24828 0 -1 4202560 7100 1253 0 0 1098 1 0 0 25 0 9 0 655141057 391462912 59225 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 89.55
Current children cumulated vsize (KiB) 382288

[startup+25.5008 s]
/proc/loadavg: 6.90 7.78 7.71 9/349 24869
/proc/meminfo: memFree=25487432/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=418084 CPUtime=191.81 cores=0-7
/proc/24854/stat : 24854 (plingeling) S 24852 24854 24828 0 -1 4202496 149077 1253 0 0 19160 21 0 0 25 0 9 0 655140888 428118016 68424 33554432000 4194304 4417452 140736700049632 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/24854/statm: 104521 68424 180 55 0 102407 0
[pid=24854/tid=24861] ppid=24852 vsize=418084 CPUtime=23.8 cores=0-7
/proc/24854/task/24861/stat : 24861 (plingeling) R 24852 24854 24828 0 -1 4202560 18048 1253 0 0 2378 2 0 0 25 0 9 0 655141057 428118016 68424 33554432000 4194304 4417452 140736700049632 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24854/tid=24862] ppid=24852 vsize=418084 CPUtime=23.79 cores=0-7
/proc/24854/task/24862/stat : 24862 (plingeling) R 24852 24854 24828 0 -1 4202560 10455 1253 0 0 2378 1 0 0 25 0 9 0 655141057 428118016 68424 33554432000 4194304 4417452 140736700049632 18446744073709551615 4341403 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=24854/tid=24863] ppid=24852 vsize=418084 CPUtime=23.79 cores=0-7
/proc/24854/task/24863/stat : 24863 (plingeling) R 24852 24854 24828 0 -1 4202560 5010 1253 0 0 2378 1 0 0 25 0 9 0 655141057 428118016 68424 33554432000 4194304 4417452 140736700049632 18446744073709551615 4346571 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=24854/tid=24864] ppid=24852 vsize=418084 CPUtime=23.52 cores=0-7
/proc/24854/task/24864/stat : 24864 (plingeling) R 24852 24854 24828 0 -1 4202560 3123 1253 0 0 2352 0 0 0 25 0 9 0 655141057 428118016 68424 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24854/tid=24865] ppid=24852 vsize=418084 CPUtime=23.79 cores=0-7
/proc/24854/task/24865/stat : 24865 (plingeling) R 24852 24854 24828 0 -1 4202560 9213 1253 0 0 2378 1 0 0 25 0 9 0 655141057 428118016 68424 33554432000 4194304 4417452 140736700049632 18446744073709551615 4265499 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=24854/tid=24866] ppid=24852 vsize=418084 CPUtime=23.8 cores=0-7
/proc/24854/task/24866/stat : 24866 (plingeling) R 24852 24854 24828 0 -1 4202560 9737 1253 0 0 2379 1 0 0 25 0 9 0 655141057 428118016 68424 33554432000 4194304 4417452 140736700049632 18446744073709551615 4252144 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=24854/tid=24867] ppid=24852 vsize=418084 CPUtime=23.78 cores=0-7
/proc/24854/task/24867/stat : 24867 (plingeling) R 24852 24854 24828 0 -1 4202560 9524 1253 0 0 2377 1 0 0 25 0 9 0 655141057 428118016 68424 33554432000 4194304 4417452 140736700049632 18446744073709551615 4275978 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=24854/tid=24868] ppid=24852 vsize=418084 CPUtime=23.78 cores=0-7
/proc/24854/task/24868/stat : 24868 (plingeling) R 24852 24854 24828 0 -1 4202560 10518 1253 0 0 2377 1 0 0 25 0 9 0 655141057 428118016 68424 33554432000 4194304 4417452 140736700049632 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 191.81
Current children cumulated vsize (KiB) 418084

[startup+51.1011 s]
/proc/loadavg: 7.39 7.81 7.73 9/349 24870
/proc/meminfo: memFree=25438332/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=473464 CPUtime=396.33 cores=0-7
/proc/24854/stat : 24854 (plingeling) S 24852 24854 24828 0 -1 4202496 176066 1253 0 0 39608 25 0 0 25 0 9 0 655140888 484827136 85003 33554432000 4194304 4417452 140736700049632 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/24854/statm: 118366 85003 180 55 0 116252 0
[pid=24854/tid=24861] ppid=24852 vsize=473464 CPUtime=49.39 cores=0-7
/proc/24854/task/24861/stat : 24861 (plingeling) R 24852 24854 24828 0 -1 4202560 20482 1253 0 0 4937 2 0 0 25 0 9 0 655141057 484827136 85003 33554432000 4194304 4417452 140736700049632 18446744073709551615 4354506 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24854/tid=24862] ppid=24852 vsize=473464 CPUtime=49.39 cores=0-7
/proc/24854/task/24862/stat : 24862 (plingeling) R 24852 24854 24828 0 -1 4202560 15077 1253 0 0 4938 1 0 0 25 0 9 0 655141057 484827136 85003 33554432000 4194304 4417452 140736700049632 18446744073709551615 4237189 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=24854/tid=24863] ppid=24852 vsize=473464 CPUtime=49.39 cores=0-7
/proc/24854/task/24863/stat : 24863 (plingeling) R 24852 24854 24828 0 -1 4202560 5299 1253 0 0 4938 1 0 0 25 0 9 0 655141057 484827136 85003 33554432000 4194304 4417452 140736700049632 18446744073709551615 4354506 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=24854/tid=24864] ppid=24852 vsize=473464 CPUtime=48.84 cores=0-7
/proc/24854/task/24864/stat : 24864 (plingeling) R 24852 24854 24828 0 -1 4202560 3123 1253 0 0 4884 0 0 0 25 0 9 0 655141057 484827136 85003 33554432000 4194304 4417452 140736700049632 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24854/tid=24865] ppid=24852 vsize=473464 CPUtime=49.39 cores=0-7
/proc/24854/task/24865/stat : 24865 (plingeling) R 24852 24854 24828 0 -1 4202560 11565 1253 0 0 4937 2 0 0 25 0 9 0 655141057 484827136 85003 33554432000 4194304 4417452 140736700049632 18446744073709551615 4281855 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=24854/tid=24866] ppid=24852 vsize=473464 CPUtime=49.39 cores=0-7
/proc/24854/task/24866/stat : 24866 (plingeling) R 24852 24854 24828 0 -1 4202560 14602 1253 0 0 4938 1 0 0 25 0 9 0 655141057 484827136 85003 33554432000 4194304 4417452 140736700049632 18446744073709551615 4357942 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=24854/tid=24867] ppid=24852 vsize=473464 CPUtime=49.38 cores=0-7
/proc/24854/task/24867/stat : 24867 (plingeling) R 24852 24854 24828 0 -1 4202560 11149 1253 0 0 4937 1 0 0 25 0 9 0 655141057 484827136 85003 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=24854/tid=24868] ppid=24852 vsize=473464 CPUtime=49.39 cores=0-7
/proc/24854/task/24868/stat : 24868 (plingeling) R 24852 24854 24828 0 -1 4202560 21320 1253 0 0 4936 3 0 0 25 0 9 0 655141057 484827136 85003 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 396.33
Current children cumulated vsize (KiB) 473464

[startup+102.313 s]
/proc/loadavg: 7.73 7.84 7.74 9/349 24871

################
# More data... #
################


[startup+4722.31 s]
/proc/loadavg: 8.00 8.02 8.00 9/349 25018
/proc/meminfo: memFree=22561936/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=3339252 CPUtime=37713 cores=0-7
/proc/24854/stat : 24854 (plingeling) S 24852 24854 24828 0 -1 4202496 4996650 1253 0 0 3770473 824 0 0 25 0 9 0 655140888 3419394048 797851 33554432000 4194304 4417452 140736700049632 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/24854/statm: 834813 797851 180 55 0 832699 0
[pid=24854/tid=24861] ppid=24852 vsize=3339252 CPUtime=4714.54 cores=0-7
/proc/24854/task/24861/stat : 24861 (plingeling) R 24852 24854 24828 0 -1 4202560 422734 1253 0 0 471392 62 0 0 25 0 9 0 655141057 3419394048 797851 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24854/tid=24862] ppid=24852 vsize=3339252 CPUtime=4720.71 cores=0-7
/proc/24854/task/24862/stat : 24862 (plingeling) R 24852 24854 24828 0 -1 4202560 991896 1253 0 0 471914 157 0 0 25 0 9 0 655141057 3419394048 797851 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=24854/tid=24863] ppid=24852 vsize=3339252 CPUtime=4719.28 cores=0-7
/proc/24854/task/24863/stat : 24863 (plingeling) R 24852 24854 24828 0 -1 4202560 518269 1253 0 0 471847 81 0 0 25 0 9 0 655141057 3419394048 797851 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=24854/tid=24864] ppid=24852 vsize=3339252 CPUtime=4684.76 cores=0-7
/proc/24854/task/24864/stat : 24864 (plingeling) R 24852 24854 24828 0 -1 4202560 926842 1253 0 0 468290 186 0 0 25 0 9 0 655141057 3419394048 797851 33554432000 4194304 4417452 140736700049632 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24854/tid=24865] ppid=24852 vsize=3339252 CPUtime=4720.7 cores=0-7
/proc/24854/task/24865/stat : 24865 (plingeling) R 24852 24854 24828 0 -1 4202560 831878 1253 0 0 471914 156 0 0 25 0 9 0 655141057 3419394048 797851 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=24854/tid=24866] ppid=24852 vsize=3339252 CPUtime=4720.71 cores=0-7
/proc/24854/task/24866/stat : 24866 (plingeling) R 24852 24854 24828 0 -1 4202560 286185 1253 0 0 472030 41 0 0 25 0 9 0 655141057 3419394048 797851 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=24854/tid=24867] ppid=24852 vsize=3339252 CPUtime=4719.53 cores=0-7
/proc/24854/task/24867/stat : 24867 (plingeling) R 24852 24854 24828 0 -1 4202560 485186 1253 0 0 471890 63 0 0 25 0 9 0 655141057 3419394048 797851 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=24854/tid=24868] ppid=24852 vsize=3339252 CPUtime=4710.97 cores=0-7
/proc/24854/task/24868/stat : 24868 (plingeling) R 24852 24854 24828 0 -1 4202560 460211 1253 0 0 471031 66 0 0 25 0 9 0 655141057 3419394048 797851 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 37713
Current children cumulated vsize (KiB) 3339252

[startup+4782.3 s]
/proc/loadavg: 8.00 8.02 8.00 9/349 25020
/proc/meminfo: memFree=22908744/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=2993588 CPUtime=38192.2 cores=0-7
/proc/24854/stat : 24854 (plingeling) S 24852 24854 24828 0 -1 4202496 5042118 1253 0 0 3818390 831 0 0 25 0 9 0 655140888 3065434112 711435 33554432000 4194304 4417452 140736700049632 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/24854/statm: 748397 711435 180 55 0 746283 0
[pid=24854/tid=24861] ppid=24852 vsize=2993588 CPUtime=4774.53 cores=0-7
/proc/24854/task/24861/stat : 24861 (plingeling) R 24852 24854 24828 0 -1 4202560 422734 1253 0 0 477391 62 0 0 25 0 9 0 655141057 3065434112 711435 33554432000 4194304 4417452 140736700049632 18446744073709551615 4353474 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24854/tid=24862] ppid=24852 vsize=2993588 CPUtime=4780.69 cores=0-7
/proc/24854/task/24862/stat : 24862 (plingeling) R 24852 24854 24828 0 -1 4202560 1026974 1253 0 0 477908 161 0 0 25 0 9 0 655141057 3065434112 711435 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=24854/tid=24863] ppid=24852 vsize=2993588 CPUtime=4779.28 cores=0-7
/proc/24854/task/24863/stat : 24863 (plingeling) R 24852 24854 24828 0 -1 4202560 522317 1253 0 0 477846 82 0 0 25 0 9 0 655141057 3065434112 711435 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=24854/tid=24864] ppid=24852 vsize=2993588 CPUtime=4744.75 cores=0-7
/proc/24854/task/24864/stat : 24864 (plingeling) R 24852 24854 24828 0 -1 4202560 926842 1253 0 0 474289 186 0 0 25 0 9 0 655141057 3065434112 711435 33554432000 4194304 4417452 140736700049632 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24854/tid=24865] ppid=24852 vsize=2993588 CPUtime=4780.69 cores=0-7
/proc/24854/task/24865/stat : 24865 (plingeling) R 24852 24854 24828 0 -1 4202560 831878 1253 0 0 477913 156 0 0 25 0 9 0 655141057 3065434112 711435 33554432000 4194304 4417452 140736700049632 18446744073709551615 4250961 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=24854/tid=24866] ppid=24852 vsize=2993588 CPUtime=4780.69 cores=0-7
/proc/24854/task/24866/stat : 24866 (plingeling) R 24852 24854 24828 0 -1 4202560 291307 1253 0 0 478027 42 0 0 25 0 9 0 655141057 3065434112 711435 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=24854/tid=24867] ppid=24852 vsize=2993588 CPUtime=4779.51 cores=0-7
/proc/24854/task/24867/stat : 24867 (plingeling) R 24852 24854 24828 0 -1 4202560 485186 1253 0 0 477888 63 0 0 25 0 9 0 655141057 3065434112 711435 33554432000 4194304 4417452 140736700049632 18446744073709551615 4366523 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=24854/tid=24868] ppid=24852 vsize=2993588 CPUtime=4770.3 cores=0-7
/proc/24854/task/24868/stat : 24868 (plingeling) R 24852 24854 24828 0 -1 4202560 461431 1253 0 0 476964 66 0 0 25 0 9 0 655141057 3065434112 711435 33554432000 4194304 4417452 140736700049632 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38192.2
Current children cumulated vsize (KiB) 2993588

[startup+4842.3 s]
/proc/loadavg: 8.05 8.03 8.00 9/349 25022
/proc/meminfo: memFree=22892348/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=3010368 CPUtime=38671.5 cores=0-7
/proc/24854/stat : 24854 (plingeling) S 24852 24854 24828 0 -1 4202496 5072788 1253 0 0 3866318 836 0 0 25 0 9 0 655140888 3082616832 715630 33554432000 4194304 4417452 140736700049632 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/24854/statm: 752592 715630 180 55 0 750478 0
[pid=24854/tid=24861] ppid=24852 vsize=3010368 CPUtime=4834.52 cores=0-7
/proc/24854/task/24861/stat : 24861 (plingeling) R 24852 24854 24828 0 -1 4202560 426843 1253 0 0 483390 62 0 0 25 0 9 0 655141057 3082616832 715630 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24854/tid=24862] ppid=24852 vsize=3010368 CPUtime=4840.69 cores=0-7
/proc/24854/task/24862/stat : 24862 (plingeling) R 24852 24854 24828 0 -1 4202560 1026974 1253 0 0 483908 161 0 0 25 0 9 0 655141057 3082616832 715630 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=24854/tid=24863] ppid=24852 vsize=3010368 CPUtime=4839.27 cores=0-7
/proc/24854/task/24863/stat : 24863 (plingeling) R 24852 24854 24828 0 -1 4202560 540890 1253 0 0 483843 84 0 0 25 0 9 0 655141057 3082616832 715630 33554432000 4194304 4417452 140736700049632 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=24854/tid=24864] ppid=24852 vsize=3010368 CPUtime=4804.75 cores=0-7
/proc/24854/task/24864/stat : 24864 (plingeling) R 24852 24854 24828 0 -1 4202560 926842 1253 0 0 480289 186 0 0 25 0 9 0 655141057 3082616832 715630 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24854/tid=24865] ppid=24852 vsize=3010368 CPUtime=4840.7 cores=0-7
/proc/24854/task/24865/stat : 24865 (plingeling) R 24852 24854 24828 0 -1 4202560 831878 1253 0 0 483913 157 0 0 25 0 9 0 655141057 3082616832 715630 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=24854/tid=24866] ppid=24852 vsize=3010368 CPUtime=4840.69 cores=0-7
/proc/24854/task/24866/stat : 24866 (plingeling) R 24852 24854 24828 0 -1 4202560 291307 1253 0 0 484027 42 0 0 25 0 9 0 655141057 3082616832 715630 33554432000 4194304 4417452 140736700049632 18446744073709551615 4365118 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=24854/tid=24867] ppid=24852 vsize=3010368 CPUtime=4839.5 cores=0-7
/proc/24854/task/24867/stat : 24867 (plingeling) R 24852 24854 24828 0 -1 4202560 485186 1253 0 0 483887 63 0 0 25 0 9 0 655141057 3082616832 715630 33554432000 4194304 4417452 140736700049632 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=24854/tid=24868] ppid=24852 vsize=3010368 CPUtime=4829.63 cores=0-7
/proc/24854/task/24868/stat : 24868 (plingeling) R 24852 24854 24828 0 -1 4202560 469419 1253 0 0 482896 67 0 0 25 0 9 0 655141057 3082616832 715630 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38671.5
Current children cumulated vsize (KiB) 3010368

[startup+4902.3 s]
/proc/loadavg: 8.07 8.04 8.00 9/349 25023
/proc/meminfo: memFree=22466900/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=3467292 CPUtime=39150.9 cores=0-7
/proc/24854/stat : 24854 (plingeling) S 24852 24854 24828 0 -1 4202496 5232405 1253 0 0 3914224 863 0 0 25 0 9 0 655140888 3550507008 829861 33554432000 4194304 4417452 140736700049632 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/24854/statm: 866823 829861 180 55 0 864709 0
[pid=24854/tid=24861] ppid=24852 vsize=3467292 CPUtime=4894.53 cores=0-7
/proc/24854/task/24861/stat : 24861 (plingeling) R 24852 24854 24828 0 -1 4202560 426843 1253 0 0 489391 62 0 0 25 0 9 0 655141057 3550507008 829861 33554432000 4194304 4417452 140736700049632 18446744073709551615 230808863054 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24854/tid=24862] ppid=24852 vsize=3467292 CPUtime=4900.69 cores=0-7
/proc/24854/task/24862/stat : 24862 (plingeling) R 24852 24854 24828 0 -1 4202560 1069377 1253 0 0 489901 168 0 0 25 0 9 0 655141057 3550507008 829861 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=24854/tid=24863] ppid=24852 vsize=3467292 CPUtime=4899.27 cores=0-7
/proc/24854/task/24863/stat : 24863 (plingeling) R 24852 24854 24828 0 -1 4202560 542302 1253 0 0 489843 84 0 0 25 0 9 0 655141057 3550507008 829861 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=24854/tid=24864] ppid=24852 vsize=3467292 CPUtime=4864.76 cores=0-7
/proc/24854/task/24864/stat : 24864 (plingeling) R 24852 24854 24828 0 -1 4202560 1025030 1253 0 0 486272 204 0 0 25 0 9 0 655141057 3550507008 829861 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24854/tid=24865] ppid=24852 vsize=3467292 CPUtime=4900.7 cores=0-7
/proc/24854/task/24865/stat : 24865 (plingeling) R 24852 24854 24828 0 -1 4202560 831878 1253 0 0 489913 157 0 0 25 0 9 0 655141057 3550507008 829861 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=24854/tid=24866] ppid=24852 vsize=3467292 CPUtime=4900.7 cores=0-7
/proc/24854/task/24866/stat : 24866 (plingeling) R 24852 24854 24828 0 -1 4202560 299499 1253 0 0 490027 43 0 0 25 0 9 0 655141057 3550507008 829861 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=24854/tid=24867] ppid=24852 vsize=3467292 CPUtime=4899.49 cores=0-7
/proc/24854/task/24867/stat : 24867 (plingeling) R 24852 24854 24828 0 -1 4202560 485186 1253 0 0 489886 63 0 0 25 0 9 0 655141057 3550507008 829861 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=24854/tid=24868] ppid=24852 vsize=3467292 CPUtime=4888.97 cores=0-7
/proc/24854/task/24868/stat : 24868 (plingeling) R 24852 24854 24828 0 -1 4202560 478841 1253 0 0 488828 69 0 0 25 0 9 0 655141057 3550507008 829861 33554432000 4194304 4417452 140736700049632 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39150.9
Current children cumulated vsize (KiB) 3467292

[startup+4962.3 s]
/proc/loadavg: 8.07 8.04 8.00 9/349 25025
/proc/meminfo: memFree=22416896/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=3484676 CPUtime=39630.2 cores=0-7
/proc/24854/stat : 24854 (plingeling) S 24852 24854 24828 0 -1 4202496 5273037 1253 0 0 3962151 869 0 0 25 0 9 0 655140888 3568308224 834207 33554432000 4194304 4417452 140736700049632 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/24854/statm: 871169 834207 180 55 0 869055 0
[pid=24854/tid=24861] ppid=24852 vsize=3484676 CPUtime=4954.53 cores=0-7
/proc/24854/task/24861/stat : 24861 (plingeling) R 24852 24854 24828 0 -1 4202560 430939 1253 0 0 495390 63 0 0 25 0 9 0 655141057 3568308224 834207 33554432000 4194304 4417452 140736700049632 18446744073709551615 4366074 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24854/tid=24862] ppid=24852 vsize=3484676 CPUtime=4960.7 cores=0-7
/proc/24854/task/24862/stat : 24862 (plingeling) R 24852 24854 24828 0 -1 4202560 1069377 1253 0 0 495901 169 0 0 25 0 9 0 655141057 3568308224 834207 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=24854/tid=24863] ppid=24852 vsize=3484676 CPUtime=4959.27 cores=0-7
/proc/24854/task/24863/stat : 24863 (plingeling) R 24852 24854 24828 0 -1 4202560 553166 1253 0 0 495842 85 0 0 25 0 9 0 655141057 3568308224 834207 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=24854/tid=24864] ppid=24852 vsize=3484676 CPUtime=4924.76 cores=0-7
/proc/24854/task/24864/stat : 24864 (plingeling) R 24852 24854 24828 0 -1 4202560 1025030 1253 0 0 492272 204 0 0 25 0 9 0 655141057 3568308224 834207 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24854/tid=24865] ppid=24852 vsize=3484676 CPUtime=4960.7 cores=0-7
/proc/24854/task/24865/stat : 24865 (plingeling) R 24852 24854 24828 0 -1 4202560 847940 1253 0 0 495910 160 0 0 25 0 9 0 655141057 3568308224 834207 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=24854/tid=24866] ppid=24852 vsize=3484676 CPUtime=4960.7 cores=0-7
/proc/24854/task/24866/stat : 24866 (plingeling) R 24852 24854 24828 0 -1 4202560 299512 1253 0 0 496027 43 0 0 25 0 9 0 655141057 3568308224 834207 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=24854/tid=24867] ppid=24852 vsize=3484676 CPUtime=4959.48 cores=0-7
/proc/24854/task/24867/stat : 24867 (plingeling) R 24852 24854 24828 0 -1 4202560 493378 1253 0 0 495884 64 0 0 25 0 9 0 655141057 3568308224 834207 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=24854/tid=24868] ppid=24852 vsize=3484676 CPUtime=4948.31 cores=0-7
/proc/24854/task/24868/stat : 24868 (plingeling) R 24852 24854 24828 0 -1 4202560 480246 1253 0 0 494762 69 0 0 25 0 9 0 655141057 3568308224 834207 33554432000 4194304 4417452 140736700049632 18446744073709551615 4366385 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39630.2
Current children cumulated vsize (KiB) 3484676



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5008.6 s]
/proc/loadavg: 8.03 8.03 8.00 9/349 25026
/proc/meminfo: memFree=22527732/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=3374148 CPUtime=40000.2 cores=0-7
/proc/24854/stat : 24854 (plingeling) S 24852 24854 24828 0 -1 4202496 5350282 1253 0 0 3999137 879 0 0 25 0 9 0 655140888 3455127552 806544 33554432000 4194304 4417452 140736700049632 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/24854/statm: 843537 806544 180 55 0 841423 0
[pid=24854/tid=24861] ppid=24852 vsize=3374148 CPUtime=5000.85 cores=0-7
/proc/24854/task/24861/stat : 24861 (plingeling) R 24852 24854 24828 0 -1 4202560 430970 1253 0 0 500022 63 0 0 25 0 9 0 655141057 3455127552 806544 33554432000 4194304 4417452 140736700049632 18446744073709551615 4263109 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24854/tid=24862] ppid=24852 vsize=3374148 CPUtime=5007.02 cores=0-7
/proc/24854/task/24862/stat : 24862 (plingeling) R 24852 24854 24828 0 -1 4202560 1069377 1253 0 0 500533 169 0 0 25 0 9 0 655141057 3455127552 806544 33554432000 4194304 4417452 140736700049632 18446744073709551615 4365062 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=24854/tid=24863] ppid=24852 vsize=3374148 CPUtime=5005.59 cores=0-7
/proc/24854/task/24863/stat : 24863 (plingeling) R 24852 24854 24828 0 -1 4202560 555926 1253 0 0 500473 86 0 0 25 0 9 0 655141057 3455127552 806544 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=24854/tid=24864] ppid=24852 vsize=3374148 CPUtime=4971.07 cores=0-7
/proc/24854/task/24864/stat : 24864 (plingeling) R 24852 24854 24828 0 -1 4202560 1025030 1253 0 0 496903 204 0 0 25 0 9 0 655141057 3455127552 806544 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24854/tid=24865] ppid=24852 vsize=3374148 CPUtime=5007.01 cores=0-7
/proc/24854/task/24865/stat : 24865 (plingeling) R 24852 24854 24828 0 -1 4202560 889280 1253 0 0 500535 166 0 0 25 0 9 0 655141057 3455127552 806544 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=24854/tid=24866] ppid=24852 vsize=3374148 CPUtime=5007.01 cores=0-7
/proc/24854/task/24866/stat : 24866 (plingeling) R 24852 24854 24828 0 -1 4202560 307735 1253 0 0 500657 44 0 0 25 0 9 0 655141057 3455127552 806544 33554432000 4194304 4417452 140736700049632 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=24854/tid=24867] ppid=24852 vsize=3374148 CPUtime=5005.78 cores=0-7
/proc/24854/task/24867/stat : 24867 (plingeling) R 24852 24854 24828 0 -1 4202560 493378 1253 0 0 500514 64 0 0 25 0 9 0 655141057 3455127552 806544 33554432000 4194304 4417452 140736700049632 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=24854/tid=24868] ppid=24852 vsize=3374148 CPUtime=4994.09 cores=0-7
/proc/24854/task/24868/stat : 24868 (plingeling) R 24852 24854 24828 0 -1 4202560 505137 1253 0 0 499338 71 0 0 25 0 9 0 655141057 3455127552 806544 33554432000 4194304 4417452 140736700049632 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.2
Current children cumulated vsize (KiB) 3374148

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5008.72 s]
/proc/loadavg: 8.03 8.03 8.00 9/349 25026
/proc/meminfo: memFree=22527732/32951124 swapFree=40554400/67111528
[pid=24854] ppid=24852 vsize=0 CPUtime=40000.3 cores=0-7
/proc/24854/stat : 24854 (plingeling) Z 24852 24854 24828 0 -1 4203548 5350283 1253 0 0 3999144 889 0 0 18 0 2 0 655140888 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/24854/statm: 0 0 0 0 0 0 0
[pid=24854/tid=24868] ppid=24852 vsize=0 CPUtime=4994.2 cores=0-7
/proc/24854/task/24868/stat : 24868 (plingeling) R 24852 24854 24828 0 -1 4203588 505137 1253 0 0 499339 81 0 0 25 0 2 0 655141057 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.3
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5008.75
CPU time (s): 40000.4
CPU user time (s): 39991.4
CPU system time (s): 8.93064
CPU usage (%): 798.61
Max. virtual memory (cumulated for all children) (KiB): 3820428

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39991.4
system time used= 8.93064
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 5351536
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 970
involuntary context switches= 193859

runsolver used 15.5326 second user time and 39.483 second system time

The end

Launcher Data 

Begin job on node104 at 2011-05-15 06:56:44
IDJOB=3372113
IDBENCH=20386
IDSOLVER=1851
FILE ID=node104/3372113-1305435404
RUNJOBID= node104-1305435404-24842
PBS_JOBID= 13324737
Free space on /tmp= 73416 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT07/industrial/anbulagan/hard-unsat/partial-5-11-u.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372113-1305435404/watcher-3372113-1305435404 -o /tmp/evaluation-result-3372113-1305435404/solver-3372113-1305435404 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3372113-1305435404.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 8e22d84b1ae625d44fbf300a32374a2d
RANDOM SEED=55414324

node104.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5293.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25760000 kB
Buffers:        304448 kB
Cached:        2409192 kB
SwapCached:     122128 kB
Active:         955544 kB
Inactive:      5966288 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25760000 kB
SwapTotal:    67111528 kB
SwapFree:     40554400 kB
Dirty:           12960 kB
Writeback:           0 kB
AnonPages:     4100128 kB
Mapped:           9996 kB
Slab:           118600 kB
PageTables:      84264 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 74803832 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73416 MiB
End job on node104 at 2011-05-15 08:20:15