Trace number 3309873

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 40000.5 5003.65

General information on the benchmark

Nameapplication/kullmann/AES/Benchmarks/
64/aes_64_3_keyfind_1.cnf
MD5SUM4738c3d0ccd52030cff8addcec6bcde9
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1404
Number of clauses7572
Sum of the clauses size29588
Maximum clause length5
Minimum clause length1
Number of clauses of size 1140
Number of clauses of size 2240
Number of clauses of size 32992
Number of clauses of size 41008
Number of clauses of size 53192
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3309873-1304013799/watcher-3309873-1304013799 -o /tmp/evaluation-result-3309873-1304013799/solver-3309873-1304013799 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3309873-1304013799.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.61 6.96 6.97 2/170 16297
/proc/meminfo: memFree=16010532/32950928 swapFree=67087032/67111528
[pid=16297] ppid=16295 vsize=9596 CPUtime=0 cores=0-7
/proc/16297/stat : 16297 (plingeling) R 16295 16297 15599 0 -1 4202496 473 1261 0 0 0 0 0 0 25 0 1 0 512977002 9826304 401 33554432000 4194304 4417332 140736585365120 18446744073709551615 239476694784 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/16297/statm: 2399 401 147 55 0 285 0

[startup+0.14979 s]
/proc/loadavg: 6.61 6.96 6.97 2/170 16297
/proc/meminfo: memFree=16010532/32950928 swapFree=67087032/67111528
[pid=16297] ppid=16295 vsize=163380 CPUtime=0.87 cores=0-7
/proc/16297/stat : 16297 (plingeling) S 16295 16297 15599 0 -1 4202496 2235 1261 0 0 87 0 0 0 18 0 9 0 512977002 167301120 2088 33554432000 4194304 4417332 140736585365120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16297/statm: 40845 2088 180 55 0 38731 0
Current children cumulated CPU time (s) 0.87
Current children cumulated vsize (KiB) 163380

[startup+0.249771 s]
/proc/loadavg: 6.61 6.96 6.97 2/170 16297
/proc/meminfo: memFree=16010532/32950928 swapFree=67087032/67111528
[pid=16297] ppid=16295 vsize=164604 CPUtime=1.64 cores=0-7
/proc/16297/stat : 16297 (plingeling) S 16295 16297 15599 0 -1 4202496 2561 1261 0 0 164 0 0 0 18 0 9 0 512977002 168554496 2393 33554432000 4194304 4417332 140736585365120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16297/statm: 41151 2393 180 55 0 39037 0
Current children cumulated CPU time (s) 1.64
Current children cumulated vsize (KiB) 164604

[startup+0.349753 s]
/proc/loadavg: 6.61 6.96 6.97 2/170 16297
/proc/meminfo: memFree=16010532/32950928 swapFree=67087032/67111528
[pid=16297] ppid=16295 vsize=166584 CPUtime=2.44 cores=0-7
/proc/16297/stat : 16297 (plingeling) S 16295 16297 15599 0 -1 4202496 3257 1261 0 0 244 0 0 0 18 0 9 0 512977002 170582016 2897 33554432000 4194304 4417332 140736585365120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16297/statm: 41646 2897 180 55 0 39532 0
Current children cumulated CPU time (s) 2.44
Current children cumulated vsize (KiB) 166584

[startup+0.700686 s]
/proc/loadavg: 6.61 6.96 6.97 2/170 16297
/proc/meminfo: memFree=16010532/32950928 swapFree=67087032/67111528
[pid=16297] ppid=16295 vsize=170896 CPUtime=5.25 cores=0-7
/proc/16297/stat : 16297 (plingeling) S 16295 16297 15599 0 -1 4202496 6208 1261 0 0 525 0 0 0 18 0 9 0 512977002 174997504 3976 33554432000 4194304 4417332 140736585365120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16297/statm: 42724 3976 180 55 0 40610 0
Current children cumulated CPU time (s) 5.25
Current children cumulated vsize (KiB) 170896

[startup+1.50053 s]
/proc/loadavg: 6.61 6.96 6.97 9/179 16311
/proc/meminfo: memFree=15990752/32950928 swapFree=67087032/67111528
[pid=16297] ppid=16295 vsize=180220 CPUtime=11.64 cores=0-7
/proc/16297/stat : 16297 (plingeling) S 16295 16297 15599 0 -1 4202496 13714 1261 0 0 1162 2 0 0 18 0 9 0 512977002 184545280 6280 33554432000 4194304 4417332 140736585365120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16297/statm: 45055 6280 180 55 0 42941 0
[pid=16297/tid=16304] ppid=16295 vsize=180220 CPUtime=1.47 cores=0-7
/proc/16297/task/16304/stat : 16304 (plingeling) R 16295 16297 15599 0 -1 4202560 1884 1261 0 0 147 0 0 0 20 0 9 0 512977004 184545280 6280 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364951 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16297/tid=16305] ppid=16295 vsize=180220 CPUtime=1.36 cores=0-7
/proc/16297/task/16305/stat : 16305 (plingeling) R 16295 16297 15599 0 -1 4202560 1552 1261 0 0 136 0 0 0 25 0 9 0 512977004 184545280 6280 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16297/tid=16306] ppid=16295 vsize=180220 CPUtime=1.47 cores=0-7
/proc/16297/task/16306/stat : 16306 (plingeling) R 16295 16297 15599 0 -1 4202560 1850 1261 0 0 147 0 0 0 19 0 9 0 512977004 184545280 6280 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364680 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16297/tid=16307] ppid=16295 vsize=180220 CPUtime=1.43 cores=0-7
/proc/16297/task/16307/stat : 16307 (plingeling) R 16295 16297 15599 0 -1 4202560 1445 1261 0 0 143 0 0 0 23 0 9 0 512977004 184545280 6280 33554432000 4194304 4417332 140736585365120 18446744073709551615 4236874 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16297/tid=16308] ppid=16295 vsize=180220 CPUtime=1.47 cores=0-7
/proc/16297/task/16308/stat : 16308 (plingeling) R 16295 16297 15599 0 -1 4202560 1231 1261 0 0 147 0 0 0 23 0 9 0 512977004 184545280 6280 33554432000 4194304 4417332 140736585365120 18446744073709551615 4251045 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16297/tid=16309] ppid=16295 vsize=180220 CPUtime=1.44 cores=0-7
/proc/16297/task/16309/stat : 16309 (plingeling) R 16295 16297 15599 0 -1 4202560 1549 1261 0 0 144 0 0 0 23 0 9 0 512977004 184545280 6280 33554432000 4194304 4417332 140736585365120 18446744073709551615 4321690 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16297/tid=16310] ppid=16295 vsize=180220 CPUtime=1.46 cores=0-7
/proc/16297/task/16310/stat : 16310 (plingeling) R 16295 16297 15599 0 -1 4202560 1440 1261 0 0 146 0 0 0 23 0 9 0 512977004 184545280 6280 33554432000 4194304 4417332 140736585365120 18446744073709551615 4355716 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16297/tid=16311] ppid=16295 vsize=180220 CPUtime=1.47 cores=0-7
/proc/16297/task/16311/stat : 16311 (plingeling) R 16295 16297 15599 0 -1 4202560 1316 1261 0 0 147 0 0 0 24 0 9 0 512977004 184545280 6280 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 11.64
Current children cumulated vsize (KiB) 180220

[startup+3.10123 s]
/proc/loadavg: 6.61 6.96 6.97 9/179 16311
/proc/meminfo: memFree=15982036/32950928 swapFree=67087032/67111528
[pid=16297] ppid=16295 vsize=186840 CPUtime=24.44 cores=0-7
/proc/16297/stat : 16297 (plingeling) S 16295 16297 15599 0 -1 4202496 25126 1261 0 0 2440 4 0 0 18 0 9 0 512977002 191324160 7935 33554432000 4194304 4417332 140736585365120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16297/statm: 46710 7935 180 55 0 44596 0
[pid=16297/tid=16304] ppid=16295 vsize=186840 CPUtime=3.07 cores=0-7
/proc/16297/task/16304/stat : 16304 (plingeling) R 16295 16297 15599 0 -1 4202560 3130 1261 0 0 307 0 0 0 25 0 9 0 512977004 191324160 7935 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16297/tid=16305] ppid=16295 vsize=186840 CPUtime=2.96 cores=0-7
/proc/16297/task/16305/stat : 16305 (plingeling) R 16295 16297 15599 0 -1 4202560 3226 1261 0 0 296 0 0 0 25 0 9 0 512977004 191324160 7935 33554432000 4194304 4417332 140736585365120 18446744073709551615 4352387 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16297/tid=16306] ppid=16295 vsize=186840 CPUtime=3.06 cores=0-7
/proc/16297/task/16306/stat : 16306 (plingeling) R 16295 16297 15599 0 -1 4202560 3093 1261 0 0 306 0 0 0 25 0 9 0 512977004 191324160 7935 33554432000 4194304 4417332 140736585365120 18446744073709551615 4251100 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16297/tid=16307] ppid=16295 vsize=186840 CPUtime=3.02 cores=0-7
/proc/16297/task/16307/stat : 16307 (plingeling) R 16295 16297 15599 0 -1 4202560 2787 1261 0 0 302 0 0 0 25 0 9 0 512977004 191324160 7935 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16297/tid=16308] ppid=16295 vsize=186840 CPUtime=3.07 cores=0-7
/proc/16297/task/16308/stat : 16308 (plingeling) R 16295 16297 15599 0 -1 4202560 2806 1261 0 0 307 0 0 0 25 0 9 0 512977004 191324160 7935 33554432000 4194304 4417332 140736585365120 18446744073709551615 4250861 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16297/tid=16309] ppid=16295 vsize=186840 CPUtime=3.03 cores=0-7
/proc/16297/task/16309/stat : 16309 (plingeling) R 16295 16297 15599 0 -1 4202560 2663 1261 0 0 303 0 0 0 25 0 9 0 512977004 191324160 7935 33554432000 4194304 4417332 140736585365120 18446744073709551615 4366455 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16297/tid=16310] ppid=16295 vsize=186840 CPUtime=3.06 cores=0-7
/proc/16297/task/16310/stat : 16310 (plingeling) R 16295 16297 15599 0 -1 4202560 3107 1261 0 0 306 0 0 0 25 0 9 0 512977004 191324160 7935 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364899 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16297/tid=16311] ppid=16295 vsize=186840 CPUtime=3.07 cores=0-7
/proc/16297/task/16311/stat : 16311 (plingeling) R 16295 16297 15599 0 -1 4202560 2867 1261 0 0 307 0 0 0 25 0 9 0 512977004 191324160 7935 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 24.44
Current children cumulated vsize (KiB) 186840

[startup+6.30061 s]
/proc/loadavg: 6.72 6.97 6.98 9/179 16311
/proc/meminfo: memFree=15975836/32950928 swapFree=67087032/67111528
[pid=16297] ppid=16295 vsize=194604 CPUtime=50.02 cores=0-7
/proc/16297/stat : 16297 (plingeling) S 16295 16297 15599 0 -1 4202496 43507 1261 0 0 4994 8 0 0 18 0 9 0 512977002 199274496 9886 33554432000 4194304 4417332 140736585365120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16297/statm: 48651 9886 180 55 0 46537 0
[pid=16297/tid=16304] ppid=16295 vsize=194604 CPUtime=6.26 cores=0-7
/proc/16297/task/16304/stat : 16304 (plingeling) R 16295 16297 15599 0 -1 4202560 5473 1261 0 0 626 0 0 0 25 0 9 0 512977004 199274496 9886 33554432000 4194304 4417332 140736585365120 18446744073709551615 4365862 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16297/tid=16305] ppid=16295 vsize=194604 CPUtime=6.16 cores=0-7
/proc/16297/task/16305/stat : 16305 (plingeling) R 16295 16297 15599 0 -1 4202560 5896 1261 0 0 615 1 0 0 25 0 9 0 512977004 199274496 9886 33554432000 4194304 4417332 140736585365120 18446744073709551615 4236549 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16297/tid=16306] ppid=16295 vsize=194604 CPUtime=6.27 cores=0-7
/proc/16297/task/16306/stat : 16306 (plingeling) R 16295 16297 15599 0 -1 4202560 5532 1261 0 0 626 1 0 0 25 0 9 0 512977004 199274496 9886 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364568 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16297/tid=16307] ppid=16295 vsize=194604 CPUtime=6.22 cores=0-7
/proc/16297/task/16307/stat : 16307 (plingeling) R 16295 16297 15599 0 -1 4202560 4680 1261 0 0 622 0 0 0 25 0 9 0 512977004 199274496 9886 33554432000 4194304 4417332 140736585365120 18446744073709551615 4357738 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16297/tid=16308] ppid=16295 vsize=194604 CPUtime=6.27 cores=0-7
/proc/16297/task/16308/stat : 16308 (plingeling) R 16295 16297 15599 0 -1 4202560 4505 1261 0 0 627 0 0 0 25 0 9 0 512977004 199274496 9886 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364590 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16297/tid=16309] ppid=16295 vsize=194604 CPUtime=6.21 cores=0-7
/proc/16297/task/16309/stat : 16309 (plingeling) R 16295 16297 15599 0 -1 4202560 4973 1261 0 0 621 0 0 0 25 0 9 0 512977004 199274496 9886 33554432000 4194304 4417332 140736585365120 18446744073709551615 4257011 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16297/tid=16310] ppid=16295 vsize=194604 CPUtime=6.26 cores=0-7
/proc/16297/task/16310/stat : 16310 (plingeling) R 16295 16297 15599 0 -1 4202560 5403 1261 0 0 625 1 0 0 25 0 9 0 512977004 199274496 9886 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16297/tid=16311] ppid=16295 vsize=194604 CPUtime=6.26 cores=0-7
/proc/16297/task/16311/stat : 16311 (plingeling) R 16295 16297 15599 0 -1 4202560 5598 1261 0 0 626 0 0 0 25 0 9 0 512977004 199274496 9886 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 50.02
Current children cumulated vsize (KiB) 194604

[startup+12.7004 s]
/proc/loadavg: 6.82 6.99 6.98 9/179 16312
/proc/meminfo: memFree=15964952/32950928 swapFree=67087032/67111528
[pid=16297] ppid=16295 vsize=215424 CPUtime=101.18 cores=0-7
/proc/16297/stat : 16297 (plingeling) S 16295 16297 15599 0 -1 4202496 78492 1261 0 0 10103 15 0 0 18 0 9 0 512977002 220594176 15308 33554432000 4194304 4417332 140736585365120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16297/statm: 53856 15308 180 55 0 51742 0
[pid=16297/tid=16304] ppid=16295 vsize=215424 CPUtime=12.67 cores=0-7
/proc/16297/task/16304/stat : 16304 (plingeling) R 16295 16297 15599 0 -1 4202560 10466 1261 0 0 1265 2 0 0 25 0 9 0 512977004 220594176 15308 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16297/tid=16305] ppid=16295 vsize=215424 CPUtime=12.56 cores=0-7
/proc/16297/task/16305/stat : 16305 (plingeling) R 16295 16297 15599 0 -1 4202560 9731 1261 0 0 1254 2 0 0 25 0 9 0 512977004 220594176 15308 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364409 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16297/tid=16306] ppid=16295 vsize=215424 CPUtime=12.67 cores=0-7
/proc/16297/task/16306/stat : 16306 (plingeling) R 16295 16297 15599 0 -1 4202560 9639 1261 0 0 1265 2 0 0 25 0 9 0 512977004 220594176 15308 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16297/tid=16307] ppid=16295 vsize=215424 CPUtime=12.62 cores=0-7
/proc/16297/task/16307/stat : 16307 (plingeling) R 16295 16297 15599 0 -1 4202560 8422 1261 0 0 1261 1 0 0 25 0 9 0 512977004 220594176 15308 33554432000 4194304 4417332 140736585365120 18446744073709551615 4365849 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16297/tid=16308] ppid=16295 vsize=215424 CPUtime=12.66 cores=0-7
/proc/16297/task/16308/stat : 16308 (plingeling) R 16295 16297 15599 0 -1 4202560 7143 1261 0 0 1266 0 0 0 25 0 9 0 512977004 220594176 15308 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16297/tid=16309] ppid=16295 vsize=215424 CPUtime=12.57 cores=0-7
/proc/16297/task/16309/stat : 16309 (plingeling) R 16295 16297 15599 0 -1 4202560 10471 1261 0 0 1256 1 0 0 25 0 9 0 512977004 220594176 15308 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16297/tid=16310] ppid=16295 vsize=215424 CPUtime=12.66 cores=0-7
/proc/16297/task/16310/stat : 16310 (plingeling) R 16295 16297 15599 0 -1 4202560 10073 1261 0 0 1264 2 0 0 25 0 9 0 512977004 220594176 15308 33554432000 4194304 4417332 140736585365120 18446744073709551615 4365991 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16297/tid=16311] ppid=16295 vsize=215424 CPUtime=12.66 cores=0-7
/proc/16297/task/16311/stat : 16311 (plingeling) R 16295 16297 15599 0 -1 4202560 11100 1261 0 0 1265 1 0 0 25 0 9 0 512977004 220594176 15308 33554432000 4194304 4417332 140736585365120 18446744073709551615 4354695 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 101.18
Current children cumulated vsize (KiB) 215424

[startup+25.5009 s]
/proc/loadavg: 7.16 7.06 7.00 9/179 16312
/proc/meminfo: memFree=15942376/32950928 swapFree=67087032/67111528
[pid=16297] ppid=16295 vsize=230896 CPUtime=203.51 cores=0-7
/proc/16297/stat : 16297 (plingeling) S 16295 16297 15599 0 -1 4202496 148690 1261 0 0 20322 29 0 0 18 0 9 0 512977002 236437504 18959 33554432000 4194304 4417332 140736585365120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16297/statm: 57724 18959 180 55 0 55610 0
[pid=16297/tid=16304] ppid=16295 vsize=230896 CPUtime=25.47 cores=0-7
/proc/16297/task/16304/stat : 16304 (plingeling) R 16295 16297 15599 0 -1 4202560 18268 1261 0 0 2543 4 0 0 25 0 9 0 512977004 236437504 18959 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364680 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16297/tid=16305] ppid=16295 vsize=230896 CPUtime=25.36 cores=0-7
/proc/16297/task/16305/stat : 16305 (plingeling) R 16295 16297 15599 0 -1 4202560 18822 1261 0 0 2533 3 0 0 25 0 9 0 512977004 236437504 18959 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16297/tid=16306] ppid=16295 vsize=230896 CPUtime=25.46 cores=0-7
/proc/16297/task/16306/stat : 16306 (plingeling) R 16295 16297 15599 0 -1 4202560 18529 1261 0 0 2543 3 0 0 25 0 9 0 512977004 236437504 18959 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364519 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16297/tid=16307] ppid=16295 vsize=230896 CPUtime=25.42 cores=0-7
/proc/16297/task/16307/stat : 16307 (plingeling) R 16295 16297 15599 0 -1 4202560 16826 1261 0 0 2539 3 0 0 25 0 9 0 512977004 236437504 18959 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16297/tid=16308] ppid=16295 vsize=230896 CPUtime=25.47 cores=0-7
/proc/16297/task/16308/stat : 16308 (plingeling) R 16295 16297 15599 0 -1 4202560 15639 1261 0 0 2545 2 0 0 25 0 9 0 512977004 236437504 18959 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16297/tid=16309] ppid=16295 vsize=230896 CPUtime=25.31 cores=0-7
/proc/16297/task/16309/stat : 16309 (plingeling) R 16295 16297 15599 0 -1 4202560 20354 1261 0 0 2528 3 0 0 25 0 9 0 512977004 236437504 18959 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16297/tid=16310] ppid=16295 vsize=230896 CPUtime=25.46 cores=0-7
/proc/16297/task/16310/stat : 16310 (plingeling) R 16295 16297 15599 0 -1 4202560 20051 1261 0 0 2542 4 0 0 25 0 9 0 512977004 236437504 18959 33554432000 4194304 4417332 140736585365120 18446744073709551615 4354410 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16297/tid=16311] ppid=16295 vsize=230896 CPUtime=25.46 cores=0-7
/proc/16297/task/16311/stat : 16311 (plingeling) R 16295 16297 15599 0 -1 4202560 18754 1261 0 0 2543 3 0 0 25 0 9 0 512977004 236437504 18959 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 203.51
Current children cumulated vsize (KiB) 230896

[startup+51.101 s]
/proc/loadavg: 7.44 7.13 7.03 9/179 16315
/proc/meminfo: memFree=15914004/32950928 swapFree=67087032/67111528
[pid=16297] ppid=16295 vsize=254108 CPUtime=408.17 cores=0-7
/proc/16297/stat : 16297 (plingeling) S 16295 16297 15599 0 -1 4202496 259599 1261 0 0 40768 49 0 0 18 0 9 0 512977002 260206592 24762 33554432000 4194304 4417332 140736585365120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16297/statm: 63527 24762 180 55 0 61413 0
[pid=16297/tid=16304] ppid=16295 vsize=254108 CPUtime=51.07 cores=0-7
/proc/16297/task/16304/stat : 16304 (plingeling) R 16295 16297 15599 0 -1 4202560 27532 1261 0 0 5102 5 0 0 25 0 9 0 512977004 260206592 24762 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16297/tid=16305] ppid=16295 vsize=254108 CPUtime=50.96 cores=0-7
/proc/16297/task/16305/stat : 16305 (plingeling) R 16295 16297 15599 0 -1 4202560 33994 1261 0 0 5090 6 0 0 25 0 9 0 512977004 260206592 24762 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364397 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16297/tid=16306] ppid=16295 vsize=254108 CPUtime=51.07 cores=0-7

################
# More data... #
################

/proc/16297/task/16307/stat : 16307 (plingeling) R 16295 16297 15599 0 -1 4202560 905225 1261 0 0 472036 158 0 0 25 0 9 0 512977004 908529664 183044 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16297/tid=16308] ppid=16295 vsize=887236 CPUtime=4722.44 cores=0-7
/proc/16297/task/16308/stat : 16308 (plingeling) R 16295 16297 15599 0 -1 4202560 661378 1261 0 0 472130 114 0 0 25 0 9 0 512977004 908529664 183044 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16297/tid=16309] ppid=16295 vsize=887236 CPUtime=4695.02 cores=0-7
/proc/16297/task/16309/stat : 16309 (plingeling) R 16295 16297 15599 0 -1 4202560 710936 1261 0 0 469384 118 0 0 25 0 9 0 512977004 908529664 183044 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16297/tid=16310] ppid=16295 vsize=887236 CPUtime=4722.44 cores=0-7
/proc/16297/task/16310/stat : 16310 (plingeling) R 16295 16297 15599 0 -1 4202560 573461 1261 0 0 472138 106 0 0 25 0 9 0 512977004 908529664 183044 33554432000 4194304 4417332 140736585365120 18446744073709551615 4356866 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16297/tid=16311] ppid=16295 vsize=887236 CPUtime=4722.45 cores=0-7
/proc/16297/task/16311/stat : 16311 (plingeling) R 16295 16297 15599 0 -1 4202560 773747 1261 0 0 472105 140 0 0 25 0 9 0 512977004 908529664 183044 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37751.5
Current children cumulated vsize (KiB) 887236

[startup+4782.3 s]
/proc/loadavg: 8.05 8.04 8.01 9/179 16454
/proc/meminfo: memFree=15288972/32950928 swapFree=67087032/67111528
[pid=16297] ppid=16295 vsize=878604 CPUtime=38231.1 cores=0-7
/proc/16297/stat : 16297 (plingeling) S 16295 16297 15599 0 -1 4202496 5420651 1261 0 0 3822146 969 0 0 18 0 9 0 512977002 899690496 180886 33554432000 4194304 4417332 140736585365120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16297/statm: 219651 180886 180 55 0 217537 0
[pid=16297/tid=16304] ppid=16295 vsize=878604 CPUtime=4782.31 cores=0-7
/proc/16297/task/16304/stat : 16304 (plingeling) R 16295 16297 15599 0 -1 4202560 449344 1261 0 0 478151 80 0 0 25 0 9 0 512977004 899690496 180886 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364680 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16297/tid=16305] ppid=16295 vsize=878604 CPUtime=4782.35 cores=0-7
/proc/16297/task/16305/stat : 16305 (plingeling) R 16295 16297 15599 0 -1 4202560 608987 1261 0 0 478121 114 0 0 25 0 9 0 512977004 899690496 180886 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16297/tid=16306] ppid=16295 vsize=878604 CPUtime=4782.45 cores=0-7
/proc/16297/task/16306/stat : 16306 (plingeling) R 16295 16297 15599 0 -1 4202560 709954 1261 0 0 478116 129 0 0 25 0 9 0 512977004 899690496 180886 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364713 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16297/tid=16307] ppid=16295 vsize=878604 CPUtime=4781.94 cores=0-7
/proc/16297/task/16307/stat : 16307 (plingeling) R 16295 16297 15599 0 -1 4202560 911455 1261 0 0 478035 159 0 0 25 0 9 0 512977004 899690496 180886 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16297/tid=16308] ppid=16295 vsize=878604 CPUtime=4782.43 cores=0-7
/proc/16297/task/16308/stat : 16308 (plingeling) R 16295 16297 15599 0 -1 4202560 666200 1261 0 0 478128 115 0 0 25 0 9 0 512977004 899690496 180886 33554432000 4194304 4417332 140736585365120 18446744073709551615 4366398 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16297/tid=16309] ppid=16295 vsize=878604 CPUtime=4754.69 cores=0-7
/proc/16297/task/16309/stat : 16309 (plingeling) R 16295 16297 15599 0 -1 4202560 717896 1261 0 0 475349 120 0 0 25 0 9 0 512977004 899690496 180886 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364753 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16297/tid=16310] ppid=16295 vsize=878604 CPUtime=4782.44 cores=0-7
/proc/16297/task/16310/stat : 16310 (plingeling) R 16295 16297 15599 0 -1 4202560 576666 1261 0 0 478137 107 0 0 25 0 9 0 512977004 899690496 180886 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16297/tid=16311] ppid=16295 vsize=878604 CPUtime=4782.44 cores=0-7
/proc/16297/task/16311/stat : 16311 (plingeling) R 16295 16297 15599 0 -1 4202560 778702 1261 0 0 478104 140 0 0 25 0 9 0 512977004 899690496 180886 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38231.2
Current children cumulated vsize (KiB) 878604

[startup+4842.31 s]
/proc/loadavg: 8.13 8.06 8.01 9/179 16455
/proc/meminfo: memFree=15317992/32950928 swapFree=67087032/67111528
[pid=16297] ppid=16295 vsize=849796 CPUtime=38710.9 cores=0-7
/proc/16297/stat : 16297 (plingeling) S 16295 16297 15599 0 -1 4202496 5458683 1261 0 0 3870111 977 0 0 18 0 9 0 512977002 870191104 173684 33554432000 4194304 4417332 140736585365120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16297/statm: 212449 173684 180 55 0 210335 0
[pid=16297/tid=16304] ppid=16295 vsize=849796 CPUtime=4842.32 cores=0-7
/proc/16297/task/16304/stat : 16304 (plingeling) R 16295 16297 15599 0 -1 4202560 452382 1261 0 0 484151 81 0 0 25 0 9 0 512977004 870191104 173684 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16297/tid=16305] ppid=16295 vsize=849796 CPUtime=4842.35 cores=0-7
/proc/16297/task/16305/stat : 16305 (plingeling) R 16295 16297 15599 0 -1 4202560 612555 1261 0 0 484121 114 0 0 25 0 9 0 512977004 870191104 173684 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16297/tid=16306] ppid=16295 vsize=849796 CPUtime=4842.46 cores=0-7
/proc/16297/task/16306/stat : 16306 (plingeling) R 16295 16297 15599 0 -1 4202560 715554 1261 0 0 484116 130 0 0 25 0 9 0 512977004 870191104 173684 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16297/tid=16307] ppid=16295 vsize=849796 CPUtime=4841.94 cores=0-7
/proc/16297/task/16307/stat : 16307 (plingeling) R 16295 16297 15599 0 -1 4202560 915145 1261 0 0 484034 160 0 0 25 0 9 0 512977004 870191104 173684 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16297/tid=16308] ppid=16295 vsize=849796 CPUtime=4842.45 cores=0-7
/proc/16297/task/16308/stat : 16308 (plingeling) R 16295 16297 15599 0 -1 4202560 672066 1261 0 0 484128 117 0 0 25 0 9 0 512977004 870191104 173684 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16297/tid=16309] ppid=16295 vsize=849796 CPUtime=4814.37 cores=0-7
/proc/16297/task/16309/stat : 16309 (plingeling) R 16295 16297 15599 0 -1 4202560 724724 1261 0 0 481316 121 0 0 25 0 9 0 512977004 870191104 173684 33554432000 4194304 4417332 140736585365120 18446744073709551615 4365869 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16297/tid=16310] ppid=16295 vsize=849796 CPUtime=4842.44 cores=0-7
/proc/16297/task/16310/stat : 16310 (plingeling) R 16295 16297 15599 0 -1 4202560 580997 1261 0 0 484136 108 0 0 25 0 9 0 512977004 870191104 173684 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16297/tid=16311] ppid=16295 vsize=849796 CPUtime=4842.45 cores=0-7
/proc/16297/task/16311/stat : 16311 (plingeling) R 16295 16297 15599 0 -1 4202560 783813 1261 0 0 484104 141 0 0 25 0 9 0 512977004 870191104 173684 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38710.9
Current children cumulated vsize (KiB) 849796

[startup+4902.3 s]
/proc/loadavg: 8.04 8.05 8.01 9/179 16457
/proc/meminfo: memFree=15343396/32950928 swapFree=67087032/67111528
[pid=16297] ppid=16295 vsize=823804 CPUtime=39190.5 cores=0-7
/proc/16297/stat : 16297 (plingeling) S 16295 16297 15599 0 -1 4202496 5494581 1261 0 0 3918069 982 0 0 18 0 9 0 512977002 843575296 167186 33554432000 4194304 4417332 140736585365120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16297/statm: 205951 167186 180 55 0 203837 0
[pid=16297/tid=16304] ppid=16295 vsize=823804 CPUtime=4902.31 cores=0-7
/proc/16297/task/16304/stat : 16304 (plingeling) R 16295 16297 15599 0 -1 4202560 454188 1261 0 0 490150 81 0 0 25 0 9 0 512977004 843575296 167186 33554432000 4194304 4417332 140736585365120 18446744073709551615 4250651 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16297/tid=16305] ppid=16295 vsize=823804 CPUtime=4902.35 cores=0-7
/proc/16297/task/16305/stat : 16305 (plingeling) R 16295 16297 15599 0 -1 4202560 617769 1261 0 0 490120 115 0 0 25 0 9 0 512977004 843575296 167186 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364531 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16297/tid=16306] ppid=16295 vsize=823804 CPUtime=4902.46 cores=0-7
/proc/16297/task/16306/stat : 16306 (plingeling) R 16295 16297 15599 0 -1 4202560 716832 1261 0 0 490115 131 0 0 25 0 9 0 512977004 843575296 167186 33554432000 4194304 4417332 140736585365120 18446744073709551615 4251028 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16297/tid=16307] ppid=16295 vsize=823804 CPUtime=4901.93 cores=0-7
/proc/16297/task/16307/stat : 16307 (plingeling) R 16295 16297 15599 0 -1 4202560 921460 1261 0 0 490032 161 0 0 25 0 9 0 512977004 843575296 167186 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16297/tid=16308] ppid=16295 vsize=823804 CPUtime=4902.44 cores=0-7
/proc/16297/task/16308/stat : 16308 (plingeling) R 16295 16297 15599 0 -1 4202560 672681 1261 0 0 490127 117 0 0 25 0 9 0 512977004 843575296 167186 33554432000 4194304 4417332 140736585365120 18446744073709551615 4366038 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16297/tid=16309] ppid=16295 vsize=823804 CPUtime=4874.04 cores=0-7
/proc/16297/task/16309/stat : 16309 (plingeling) R 16295 16297 15599 0 -1 4202560 732096 1261 0 0 487283 121 0 0 25 0 9 0 512977004 843575296 167186 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16297/tid=16310] ppid=16295 vsize=823804 CPUtime=4902.44 cores=0-7
/proc/16297/task/16310/stat : 16310 (plingeling) R 16295 16297 15599 0 -1 4202560 588837 1261 0 0 490135 109 0 0 25 0 9 0 512977004 843575296 167186 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364800 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16297/tid=16311] ppid=16295 vsize=823804 CPUtime=4902.45 cores=0-7
/proc/16297/task/16311/stat : 16311 (plingeling) R 16295 16297 15599 0 -1 4202560 789271 1261 0 0 490103 142 0 0 25 0 9 0 512977004 843575296 167186 33554432000 4194304 4417332 140736585365120 18446744073709551615 4366011 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39190.5
Current children cumulated vsize (KiB) 823804

[startup+4962.3 s]
/proc/loadavg: 8.01 8.04 8.00 9/179 16459
/proc/meminfo: memFree=15333716/32950928 swapFree=67087032/67111528
[pid=16297] ppid=16295 vsize=834000 CPUtime=39670.2 cores=0-7
/proc/16297/stat : 16297 (plingeling) S 16295 16297 15599 0 -1 4202496 5525564 1261 0 0 3966034 986 0 0 18 0 9 0 512977002 854016000 169735 33554432000 4194304 4417332 140736585365120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16297/statm: 208500 169735 180 55 0 206386 0
[pid=16297/tid=16304] ppid=16295 vsize=834000 CPUtime=4962.31 cores=0-7
/proc/16297/task/16304/stat : 16304 (plingeling) R 16295 16297 15599 0 -1 4202560 456596 1261 0 0 496150 81 0 0 25 0 9 0 512977004 854016000 169735 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16297/tid=16305] ppid=16295 vsize=834000 CPUtime=4962.35 cores=0-7
/proc/16297/task/16305/stat : 16305 (plingeling) R 16295 16297 15599 0 -1 4202560 621651 1261 0 0 496119 116 0 0 25 0 9 0 512977004 854016000 169735 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364519 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16297/tid=16306] ppid=16295 vsize=834000 CPUtime=4962.46 cores=0-7
/proc/16297/task/16306/stat : 16306 (plingeling) R 16295 16297 15599 0 -1 4202560 720666 1261 0 0 496115 131 0 0 25 0 9 0 512977004 854016000 169735 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16297/tid=16307] ppid=16295 vsize=834000 CPUtime=4961.93 cores=0-7
/proc/16297/task/16307/stat : 16307 (plingeling) R 16295 16297 15599 0 -1 4202560 925199 1261 0 0 496031 162 0 0 25 0 9 0 512977004 854016000 169735 33554432000 4194304 4417332 140736585365120 18446744073709551615 4250623 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16297/tid=16308] ppid=16295 vsize=834000 CPUtime=4962.44 cores=0-7
/proc/16297/task/16308/stat : 16308 (plingeling) R 16295 16297 15599 0 -1 4202560 676986 1261 0 0 496127 117 0 0 25 0 9 0 512977004 854016000 169735 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364671 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16297/tid=16309] ppid=16295 vsize=834000 CPUtime=4933.71 cores=0-7
/proc/16297/task/16309/stat : 16309 (plingeling) R 16295 16297 15599 0 -1 4202560 735568 1261 0 0 493249 122 0 0 25 0 9 0 512977004 854016000 169735 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16297/tid=16310] ppid=16295 vsize=834000 CPUtime=4962.45 cores=0-7
/proc/16297/task/16310/stat : 16310 (plingeling) R 16295 16297 15599 0 -1 4202560 594661 1261 0 0 496135 110 0 0 25 0 9 0 512977004 854016000 169735 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364680 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16297/tid=16311] ppid=16295 vsize=834000 CPUtime=4962.46 cores=0-7
/proc/16297/task/16311/stat : 16311 (plingeling) R 16295 16297 15599 0 -1 4202560 792790 1261 0 0 496103 143 0 0 25 0 9 0 512977004 854016000 169735 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364713 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39670.2
Current children cumulated vsize (KiB) 834000



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5003.6 s]
/proc/loadavg: 8.01 8.03 8.00 9/179 16460
/proc/meminfo: memFree=15290192/32950928 swapFree=67087032/67111528
[pid=16297] ppid=16295 vsize=876840 CPUtime=40000.4 cores=0-7
/proc/16297/stat : 16297 (plingeling) S 16295 16297 15599 0 -1 4202496 5559584 1261 0 0 3999051 992 0 0 18 0 9 0 512977002 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16297/statm: 219210 180445 180 55 0 217096 0
[pid=16297/tid=16304] ppid=16295 vsize=876840 CPUtime=5003.61 cores=0-7
/proc/16297/task/16304/stat : 16304 (plingeling) R 16295 16297 15599 0 -1 4202560 456596 1261 0 0 500280 81 0 0 25 0 9 0 512977004 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16297/tid=16305] ppid=16295 vsize=876840 CPUtime=5003.66 cores=0-7
/proc/16297/task/16305/stat : 16305 (plingeling) R 16295 16297 15599 0 -1 4202560 622947 1261 0 0 500250 116 0 0 25 0 9 0 512977004 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364452 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16297/tid=16306] ppid=16295 vsize=876840 CPUtime=5003.77 cores=0-7
/proc/16297/task/16306/stat : 16306 (plingeling) R 16295 16297 15599 0 -1 4202560 731227 1261 0 0 500243 134 0 0 25 0 9 0 512977004 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 4353476 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16297/tid=16307] ppid=16295 vsize=876840 CPUtime=5003.23 cores=0-7
/proc/16297/task/16307/stat : 16307 (plingeling) R 16295 16297 15599 0 -1 4202560 931561 1261 0 0 500160 163 0 0 25 0 9 0 512977004 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16297/tid=16308] ppid=16295 vsize=876840 CPUtime=5003.75 cores=0-7
/proc/16297/task/16308/stat : 16308 (plingeling) R 16295 16297 15599 0 -1 4202560 681906 1261 0 0 500257 118 0 0 25 0 9 0 512977004 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16297/tid=16309] ppid=16295 vsize=876840 CPUtime=4974.79 cores=0-7
/proc/16297/task/16309/stat : 16309 (plingeling) R 16295 16297 15599 0 -1 4202560 739040 1261 0 0 497356 123 0 0 25 0 9 0 512977004 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 4365963 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16297/tid=16310] ppid=16295 vsize=876840 CPUtime=5003.76 cores=0-7
/proc/16297/task/16310/stat : 16310 (plingeling) R 16295 16297 15599 0 -1 4202560 594661 1261 0 0 500266 110 0 0 25 0 9 0 512977004 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16297/tid=16311] ppid=16295 vsize=876840 CPUtime=5003.76 cores=0-7
/proc/16297/task/16311/stat : 16311 (plingeling) R 16295 16297 15599 0 -1 4202560 800199 1261 0 0 500233 143 0 0 25 0 9 0 512977004 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364473 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 876840

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5003.6 s]
/proc/loadavg: 8.01 8.03 8.00 9/179 16460
/proc/meminfo: memFree=15290192/32950928 swapFree=67087032/67111528
[pid=16297] ppid=16295 vsize=876840 CPUtime=40000.4 cores=0-7
/proc/16297/stat : 16297 (plingeling) S 16295 16297 15599 0 -1 4202496 5559584 1261 0 0 3999051 992 0 0 18 0 9 0 512977002 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16297/statm: 219210 180445 180 55 0 217096 0
[pid=16297/tid=16304] ppid=16295 vsize=876840 CPUtime=5003.61 cores=0-7
/proc/16297/task/16304/stat : 16304 (plingeling) R 16295 16297 15599 0 -1 4202560 456596 1261 0 0 500280 81 0 0 25 0 9 0 512977004 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16297/tid=16305] ppid=16295 vsize=876840 CPUtime=5003.66 cores=0-7
/proc/16297/task/16305/stat : 16305 (plingeling) R 16295 16297 15599 0 -1 4202560 622947 1261 0 0 500250 116 0 0 25 0 9 0 512977004 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364452 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16297/tid=16306] ppid=16295 vsize=876840 CPUtime=5003.77 cores=0-7
/proc/16297/task/16306/stat : 16306 (plingeling) R 16295 16297 15599 0 -1 4202560 731227 1261 0 0 500243 134 0 0 25 0 9 0 512977004 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 4353476 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16297/tid=16307] ppid=16295 vsize=876840 CPUtime=5003.23 cores=0-7
/proc/16297/task/16307/stat : 16307 (plingeling) R 16295 16297 15599 0 -1 4202560 931561 1261 0 0 500160 163 0 0 25 0 9 0 512977004 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16297/tid=16308] ppid=16295 vsize=876840 CPUtime=5003.75 cores=0-7
/proc/16297/task/16308/stat : 16308 (plingeling) R 16295 16297 15599 0 -1 4202560 681906 1261 0 0 500257 118 0 0 25 0 9 0 512977004 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16297/tid=16309] ppid=16295 vsize=876840 CPUtime=4974.79 cores=0-7
/proc/16297/task/16309/stat : 16309 (plingeling) R 16295 16297 15599 0 -1 4202560 739040 1261 0 0 497356 123 0 0 25 0 9 0 512977004 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 4365963 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16297/tid=16310] ppid=16295 vsize=876840 CPUtime=5003.76 cores=0-7
/proc/16297/task/16310/stat : 16310 (plingeling) R 16295 16297 15599 0 -1 4202560 594661 1261 0 0 500266 110 0 0 25 0 9 0 512977004 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16297/tid=16311] ppid=16295 vsize=876840 CPUtime=5003.76 cores=0-7
/proc/16297/task/16311/stat : 16311 (plingeling) R 16295 16297 15599 0 -1 4202560 800199 1261 0 0 500233 143 0 0 25 0 9 0 512977004 897884160 180445 33554432000 4194304 4417332 140736585365120 18446744073709551615 4364473 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 876840

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5003.65
CPU time (s): 40000.5
CPU user time (s): 39990.6
CPU system time (s): 9.96149
CPU usage (%): 799.427
Max. virtual memory (cumulated for all children) (KiB): 930436

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39990.6
system time used= 9.96148
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 5560845
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 278
involuntary context switches= 190734

runsolver used 7.45287 second user time and 19.845 second system time

The end

Launcher Data 

Begin job on node148 at 2011-04-28 20:03:19
IDJOB=3309873
IDBENCH=83421
IDSOLVER=1695
FILE ID=node148/3309873-1304013799
RUNJOBID= node148-1304002541-15613
PBS_JOBID= 13168795
Free space on /tmp= 71436 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/application/kullmann/AES/Benchmarks/64/aes_64_3_keyfind_1.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3309873-1304013799/watcher-3309873-1304013799 -o /tmp/evaluation-result-3309873-1304013799/solver-3309873-1304013799 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3309873-1304013799.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 4738c3d0ccd52030cff8addcec6bcde9
RANDOM SEED=1707869219

node148.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      16010812 kB
Buffers:       2083952 kB
Cached:       14242792 kB
SwapCached:       5264 kB
Active:        6845664 kB
Inactive:      9525376 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      16010812 kB
SwapTotal:    67111528 kB
SwapFree:     67087032 kB
Dirty:             428 kB
Writeback:           0 kB
AnonPages:       41100 kB
Mapped:          13692 kB
Slab:           503864 kB
PageTables:       5072 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   185048 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71432 MiB
End job on node148 at 2011-04-28 21:26:45