Trace number 3394614

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.6 5011.58

General information on the benchmark

Namecrafted/kullmann/VanDerWaerden/
VanderWaerden3k/VanDerWaerden_2-3-20_389.cnf
MD5SUMf105570f47bdca662c2d3d185f8dc1f8
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables389
Number of clauses41426
Sum of the clauses size188708
Maximum clause length20
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 337636
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 53790

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394614-1306100331/watcher-3394614-1306100331 -o /tmp/evaluation-result-3394614-1306100331/solver-3394614-1306100331 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394614-1306100331.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.40 7.93 7.99 2/528 9270
/proc/meminfo: memFree=15527136/32951124 swapFree=51873512/67111528
[pid=9270] ppid=9268 vsize=10008 CPUtime=0 cores=0-7
/proc/9270/stat : 9270 (plingeling) R 9268 9270 9244 0 -1 4202496 594 1253 0 0 0 0 0 0 24 0 1 0 702449400 10248192 525 33554432000 4194304 4417452 140736614271856 18446744073709551615 212039655168 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/9270/statm: 2502 525 148 55 0 388 0

[startup+0.022065 s]
/proc/loadavg: 7.40 7.93 7.99 2/528 9270
/proc/meminfo: memFree=15527136/32951124 swapFree=51873512/67111528
[pid=9270] ppid=9268 vsize=13300 CPUtime=0.01 cores=0-7
/proc/9270/stat : 9270 (plingeling) R 9268 9270 9244 0 -1 4202496 1479 1253 0 0 1 0 0 0 24 0 1 0 702449400 13619200 1332 33554432000 4194304 4417452 140736614271856 18446744073709551615 212039655168 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/9270/statm: 3325 1332 149 55 0 1211 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 13300

[startup+0.101091 s]
/proc/loadavg: 7.40 7.93 7.99 2/528 9270
/proc/meminfo: memFree=15527136/32951124 swapFree=51873512/67111528
[pid=9270] ppid=9268 vsize=29780 CPUtime=0.09 cores=0-7
/proc/9270/stat : 9270 (plingeling) R 9268 9270 9244 0 -1 4202496 5732 1253 0 0 8 1 0 0 24 0 1 0 702449400 30494720 5469 33554432000 4194304 4417452 140736614271856 18446744073709551615 4231532 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/9270/statm: 7445 5469 149 55 0 5331 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 29780

[startup+0.301017 s]
/proc/loadavg: 7.40 7.93 7.99 2/528 9270
/proc/meminfo: memFree=15527136/32951124 swapFree=51873512/67111528
[pid=9270] ppid=9268 vsize=246856 CPUtime=1.6 cores=0-7
/proc/9270/stat : 9270 (plingeling) S 9268 9270 9244 0 -1 4202496 10355 1253 0 0 159 1 0 0 25 0 9 0 702449400 252780544 6984 33554432000 4194304 4417452 140736614271856 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9270/statm: 61714 6984 179 55 0 59600 0
Current children cumulated CPU time (s) 1.6
Current children cumulated vsize (KiB) 246856

[startup+0.700941 s]
/proc/loadavg: 7.40 7.93 7.99 2/528 9270
/proc/meminfo: memFree=15527136/32951124 swapFree=51873512/67111528
[pid=9270] ppid=9268 vsize=246856 CPUtime=4.8 cores=0-7
/proc/9270/stat : 9270 (plingeling) S 9268 9270 9244 0 -1 4202496 10357 1253 0 0 479 1 0 0 25 0 9 0 702449400 252780544 6986 33554432000 4194304 4417452 140736614271856 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9270/statm: 61714 6986 179 55 0 59600 0
Current children cumulated CPU time (s) 4.8
Current children cumulated vsize (KiB) 246856

[startup+1.50081 s]
/proc/loadavg: 7.40 7.93 7.99 9/537 9284
/proc/meminfo: memFree=15498784/32951124 swapFree=51873512/67111528
[pid=9270] ppid=9268 vsize=251448 CPUtime=11.17 cores=0-7
/proc/9270/stat : 9270 (plingeling) S 9268 9270 9244 0 -1 4202496 11511 1253 0 0 1116 1 0 0 25 0 9 0 702449400 257482752 8140 33554432000 4194304 4417452 140736614271856 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9270/statm: 62862 8140 179 55 0 60748 0
[pid=9270/tid=9277] ppid=9268 vsize=251448 CPUtime=1.39 cores=0-7
/proc/9270/task/9277/stat : 9277 (plingeling) R 9268 9270 9244 0 -1 4202560 812 1253 0 0 139 0 0 0 19 0 9 0 702449411 257482752 8140 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9270/tid=9278] ppid=9268 vsize=251448 CPUtime=1.39 cores=0-7
/proc/9270/task/9278/stat : 9278 (plingeling) R 9268 9270 9244 0 -1 4202560 950 1253 0 0 139 0 0 0 25 0 9 0 702449411 257482752 8140 33554432000 4194304 4417452 140736614271856 18446744073709551615 4343764 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9270/tid=9279] ppid=9268 vsize=251448 CPUtime=1.35 cores=0-7
/proc/9270/task/9279/stat : 9279 (plingeling) R 9268 9270 9244 0 -1 4202560 420 1253 0 0 135 0 0 0 25 0 9 0 702449411 257482752 8140 33554432000 4194304 4417452 140736614271856 18446744073709551615 4320963 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9270/tid=9280] ppid=9268 vsize=251448 CPUtime=1.39 cores=0-7
/proc/9270/task/9280/stat : 9280 (plingeling) R 9268 9270 9244 0 -1 4202560 489 1253 0 0 139 0 0 0 19 0 9 0 702449411 257482752 8140 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9270/tid=9281] ppid=9268 vsize=251448 CPUtime=1.38 cores=0-7
/proc/9270/task/9281/stat : 9281 (plingeling) R 9268 9270 9244 0 -1 4202560 1052 1253 0 0 138 0 0 0 20 0 9 0 702449411 257482752 8140 33554432000 4194304 4417452 140736614271856 18446744073709551615 4250771 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9270/tid=9282] ppid=9268 vsize=251448 CPUtime=1.38 cores=0-7
/proc/9270/task/9282/stat : 9282 (plingeling) R 9268 9270 9244 0 -1 4202560 804 1253 0 0 138 0 0 0 19 0 9 0 702449411 257482752 8140 33554432000 4194304 4417452 140736614271856 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9270/tid=9283] ppid=9268 vsize=251448 CPUtime=1.38 cores=0-7
/proc/9270/task/9283/stat : 9283 (plingeling) R 9268 9270 9244 0 -1 4202560 618 1253 0 0 138 0 0 0 25 0 9 0 702449411 257482752 8140 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364687 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9270/tid=9284] ppid=9268 vsize=251448 CPUtime=1.38 cores=0-7
/proc/9270/task/9284/stat : 9284 (plingeling) R 9268 9270 9244 0 -1 4202560 618 1253 0 0 138 0 0 0 19 0 9 0 702449412 257482752 8140 33554432000 4194304 4417452 140736614271856 18446744073709551615 4353557 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 11.17
Current children cumulated vsize (KiB) 251448

[startup+3.10052 s]
/proc/loadavg: 7.40 7.93 7.99 9/537 9284
/proc/meminfo: memFree=15488244/32951124 swapFree=51873512/67111528
[pid=9270] ppid=9268 vsize=263352 CPUtime=23.95 cores=0-7
/proc/9270/stat : 9270 (plingeling) S 9268 9270 9244 0 -1 4202496 14949 1253 0 0 2393 2 0 0 25 0 9 0 702449400 269672448 11126 33554432000 4194304 4417452 140736614271856 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9270/statm: 65838 11126 179 55 0 63724 0
[pid=9270/tid=9277] ppid=9268 vsize=263352 CPUtime=2.99 cores=0-7
/proc/9270/task/9277/stat : 9277 (plingeling) R 9268 9270 9244 0 -1 4202560 1139 1253 0 0 299 0 0 0 25 0 9 0 702449411 269672448 11126 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9270/tid=9278] ppid=9268 vsize=263352 CPUtime=2.99 cores=0-7
/proc/9270/task/9278/stat : 9278 (plingeling) R 9268 9270 9244 0 -1 4202560 1162 1253 0 0 299 0 0 0 25 0 9 0 702449411 269672448 11126 33554432000 4194304 4417452 140736614271856 18446744073709551615 4356796 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9270/tid=9279] ppid=9268 vsize=263352 CPUtime=2.93 cores=0-7
/proc/9270/task/9279/stat : 9279 (plingeling) R 9268 9270 9244 0 -1 4202560 420 1253 0 0 293 0 0 0 25 0 9 0 702449411 269672448 11126 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364542 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9270/tid=9280] ppid=9268 vsize=263352 CPUtime=2.98 cores=0-7
/proc/9270/task/9280/stat : 9280 (plingeling) R 9268 9270 9244 0 -1 4202560 1798 1253 0 0 298 0 0 0 24 0 9 0 702449411 269672448 11126 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9270/tid=9281] ppid=9268 vsize=263352 CPUtime=2.98 cores=0-7
/proc/9270/task/9281/stat : 9281 (plingeling) R 9268 9270 9244 0 -1 4202560 1738 1253 0 0 298 0 0 0 24 0 9 0 702449411 269672448 11126 33554432000 4194304 4417452 140736614271856 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9270/tid=9282] ppid=9268 vsize=263352 CPUtime=2.98 cores=0-7
/proc/9270/task/9282/stat : 9282 (plingeling) R 9268 9270 9244 0 -1 4202560 1139 1253 0 0 298 0 0 0 24 0 9 0 702449411 269672448 11126 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364805 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9270/tid=9283] ppid=9268 vsize=263352 CPUtime=2.98 cores=0-7
/proc/9270/task/9283/stat : 9283 (plingeling) R 9268 9270 9244 0 -1 4202560 815 1253 0 0 298 0 0 0 25 0 9 0 702449411 269672448 11126 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9270/tid=9284] ppid=9268 vsize=263352 CPUtime=2.98 cores=0-7
/proc/9270/task/9284/stat : 9284 (plingeling) R 9268 9270 9244 0 -1 4202560 990 1253 0 0 298 0 0 0 25 0 9 0 702449412 269672448 11126 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364597 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 23.95
Current children cumulated vsize (KiB) 263352

[startup+6.30095 s]
/proc/loadavg: 7.45 7.94 7.99 9/537 9284
/proc/meminfo: memFree=15462196/32951124 swapFree=51873512/67111528
[pid=9270] ppid=9268 vsize=291868 CPUtime=49.5 cores=0-7
/proc/9270/stat : 9270 (plingeling) S 9268 9270 9244 0 -1 4202496 25380 1253 0 0 4946 4 0 0 25 0 9 0 702449400 298872832 18255 33554432000 4194304 4417452 140736614271856 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9270/statm: 72967 18255 179 55 0 70853 0
[pid=9270/tid=9277] ppid=9268 vsize=291868 CPUtime=6.18 cores=0-7
/proc/9270/task/9277/stat : 9277 (plingeling) R 9268 9270 9244 0 -1 4202560 1651 1253 0 0 618 0 0 0 25 0 9 0 702449411 298872832 18255 33554432000 4194304 4417452 140736614271856 18446744073709551615 4235565 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9270/tid=9278] ppid=9268 vsize=291868 CPUtime=6.19 cores=0-7
/proc/9270/task/9278/stat : 9278 (plingeling) R 9268 9270 9244 0 -1 4202560 2440 1253 0 0 619 0 0 0 25 0 9 0 702449411 298872832 18255 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9270/tid=9279] ppid=9268 vsize=291868 CPUtime=6.07 cores=0-7
/proc/9270/task/9279/stat : 9279 (plingeling) R 9268 9270 9244 0 -1 4202560 1373 1253 0 0 607 0 0 0 25 0 9 0 702449411 298872832 18255 33554432000 4194304 4417452 140736614271856 18446744073709551615 4366533 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9270/tid=9280] ppid=9268 vsize=291868 CPUtime=6.18 cores=0-7
/proc/9270/task/9280/stat : 9280 (plingeling) R 9268 9270 9244 0 -1 4202560 4554 1253 0 0 618 0 0 0 25 0 9 0 702449411 298872832 18255 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9270/tid=9281] ppid=9268 vsize=291868 CPUtime=6.18 cores=0-7
/proc/9270/task/9281/stat : 9281 (plingeling) R 9268 9270 9244 0 -1 4202560 2977 1253 0 0 618 0 0 0 25 0 9 0 702449411 298872832 18255 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9270/tid=9282] ppid=9268 vsize=291868 CPUtime=6.18 cores=0-7
/proc/9270/task/9282/stat : 9282 (plingeling) R 9268 9270 9244 0 -1 4202560 2789 1253 0 0 618 0 0 0 25 0 9 0 702449411 298872832 18255 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9270/tid=9283] ppid=9268 vsize=291868 CPUtime=6.18 cores=0-7
/proc/9270/task/9283/stat : 9283 (plingeling) R 9268 9270 9244 0 -1 4202560 1649 1253 0 0 618 0 0 0 25 0 9 0 702449411 298872832 18255 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9270/tid=9284] ppid=9268 vsize=291868 CPUtime=6.18 cores=0-7
/proc/9270/task/9284/stat : 9284 (plingeling) R 9268 9270 9244 0 -1 4202560 2199 1253 0 0 618 0 0 0 25 0 9 0 702449412 298872832 18255 33554432000 4194304 4417452 140736614271856 18446744073709551615 4365984 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 49.5
Current children cumulated vsize (KiB) 291868

[startup+12.7008 s]
/proc/loadavg: 7.50 7.94 7.99 9/537 9284
/proc/meminfo: memFree=15436172/32951124 swapFree=51873512/67111528
[pid=9270] ppid=9268 vsize=315872 CPUtime=100.57 cores=0-7
/proc/9270/stat : 9270 (plingeling) S 9268 9270 9244 0 -1 4202496 33575 1253 0 0 10052 5 0 0 25 0 9 0 702449400 323452928 24256 33554432000 4194304 4417452 140736614271856 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9270/statm: 78968 24256 179 55 0 76854 0
[pid=9270/tid=9277] ppid=9268 vsize=315872 CPUtime=12.58 cores=0-7
/proc/9270/task/9277/stat : 9277 (plingeling) R 9268 9270 9244 0 -1 4202560 2000 1253 0 0 1258 0 0 0 25 0 9 0 702449411 323452928 24256 33554432000 4194304 4417452 140736614271856 18446744073709551615 4276054 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9270/tid=9278] ppid=9268 vsize=315872 CPUtime=12.58 cores=0-7
/proc/9270/task/9278/stat : 9278 (plingeling) R 9268 9270 9244 0 -1 4202560 3435 1253 0 0 1258 0 0 0 25 0 9 0 702449411 323452928 24256 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9270/tid=9279] ppid=9268 vsize=315872 CPUtime=12.36 cores=0-7
/proc/9270/task/9279/stat : 9279 (plingeling) R 9268 9270 9244 0 -1 4202560 2145 1253 0 0 1236 0 0 0 25 0 9 0 702449411 323452928 24256 33554432000 4194304 4417452 140736614271856 18446744073709551615 4250447 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9270/tid=9280] ppid=9268 vsize=315872 CPUtime=12.57 cores=0-7
/proc/9270/task/9280/stat : 9280 (plingeling) R 9268 9270 9244 0 -1 4202560 7911 1253 0 0 1256 1 0 0 25 0 9 0 702449411 323452928 24256 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9270/tid=9281] ppid=9268 vsize=315872 CPUtime=12.58 cores=0-7
/proc/9270/task/9281/stat : 9281 (plingeling) R 9268 9270 9244 0 -1 4202560 3409 1253 0 0 1258 0 0 0 25 0 9 0 702449411 323452928 24256 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9270/tid=9282] ppid=9268 vsize=315872 CPUtime=12.58 cores=0-7
/proc/9270/task/9282/stat : 9282 (plingeling) R 9268 9270 9244 0 -1 4202560 3785 1253 0 0 1258 0 0 0 25 0 9 0 702449411 323452928 24256 33554432000 4194304 4417452 140736614271856 18446744073709551615 4251063 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9270/tid=9283] ppid=9268 vsize=315872 CPUtime=12.58 cores=0-7
/proc/9270/task/9283/stat : 9283 (plingeling) R 9268 9270 9244 0 -1 4202560 2717 1253 0 0 1258 0 0 0 25 0 9 0 702449411 323452928 24256 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9270/tid=9284] ppid=9268 vsize=315872 CPUtime=12.58 cores=0-7
/proc/9270/task/9284/stat : 9284 (plingeling) R 9268 9270 9244 0 -1 4202560 2425 1253 0 0 1258 0 0 0 25 0 9 0 702449412 323452928 24256 33554432000 4194304 4417452 140736614271856 18446744073709551615 4365042 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 100.57
Current children cumulated vsize (KiB) 315872

[startup+25.5005 s]
/proc/loadavg: 7.61 7.94 7.99 9/537 9284
/proc/meminfo: memFree=15420276/32951124 swapFree=51873512/67111528
[pid=9270] ppid=9268 vsize=328616 CPUtime=202.74 cores=0-7
/proc/9270/stat : 9270 (plingeling) S 9268 9270 9244 0 -1 4202496 51663 1253 0 0 20266 8 0 0 25 0 9 0 702449400 336502784 27452 33554432000 4194304 4417452 140736614271856 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9270/statm: 82154 27452 179 55 0 80040 0
[pid=9270/tid=9277] ppid=9268 vsize=328616 CPUtime=25.38 cores=0-7
/proc/9270/task/9277/stat : 9277 (plingeling) R 9268 9270 9244 0 -1 4202560 2000 1253 0 0 2538 0 0 0 25 0 9 0 702449411 336502784 27452 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9270/tid=9278] ppid=9268 vsize=328616 CPUtime=25.38 cores=0-7
/proc/9270/task/9278/stat : 9278 (plingeling) R 9268 9270 9244 0 -1 4202560 6508 1253 0 0 2537 1 0 0 25 0 9 0 702449411 336502784 27452 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9270/tid=9279] ppid=9268 vsize=328616 CPUtime=24.93 cores=0-7
/proc/9270/task/9279/stat : 9279 (plingeling) R 9268 9270 9244 0 -1 4202560 2907 1253 0 0 2493 0 0 0 25 0 9 0 702449411 336502784 27452 33554432000 4194304 4417452 140736614271856 18446744073709551615 4250447 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9270/tid=9280] ppid=9268 vsize=328616 CPUtime=25.37 cores=0-7
/proc/9270/task/9280/stat : 9280 (plingeling) R 9268 9270 9244 0 -1 4202560 18555 1253 0 0 2534 3 0 0 25 0 9 0 702449411 336502784 27452 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364749 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9270/tid=9281] ppid=9268 vsize=328616 CPUtime=25.37 cores=0-7
/proc/9270/task/9281/stat : 9281 (plingeling) R 9268 9270 9244 0 -1 4202560 5726 1253 0 0 2537 0 0 0 25 0 9 0 702449411 336502784 27452 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9270/tid=9282] ppid=9268 vsize=328616 CPUtime=25.38 cores=0-7
/proc/9270/task/9282/stat : 9282 (plingeling) R 9268 9270 9244 0 -1 4202560 4215 1253 0 0 2538 0 0 0 25 0 9 0 702449411 336502784 27452 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9270/tid=9283] ppid=9268 vsize=328616 CPUtime=25.38 cores=0-7
/proc/9270/task/9283/stat : 9283 (plingeling) R 9268 9270 9244 0 -1 4202560 3234 1253 0 0 2538 0 0 0 25 0 9 0 702449411 336502784 27452 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9270/tid=9284] ppid=9268 vsize=328616 CPUtime=25.38 cores=0-7
/proc/9270/task/9284/stat : 9284 (plingeling) R 9268 9270 9244 0 -1 4202560 2770 1253 0 0 2538 0 0 0 25 0 9 0 702449412 336502784 27452 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 202.74
Current children cumulated vsize (KiB) 328616

[startup+51.1009 s]
/proc/loadavg: 7.74 7.94 7.99 9/537 9285
/proc/meminfo: memFree=15404632/32951124 swapFree=51873512/67111528
[pid=9270] ppid=9268 vsize=341140 CPUtime=407.1 cores=0-7
/proc/9270/stat : 9270 (plingeling) S 9268 9270 9244 0 -1 4202496 135261 1253 0 0 40685 25 0 0 25 0 9 0 702449400 349327360 30583 33554432000 4194304 4417452 140736614271856 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9270/statm: 85285 30583 179 55 0 83171 0
[pid=9270/tid=9277] ppid=9268 vsize=341140 CPUtime=50.98 cores=0-7
/proc/9270/task/9277/stat : 9277 (plingeling) R 9268 9270 9244 0 -1 4202560 2000 1253 0 0 5098 0 0 0 25 0 9 0 702449411 349327360 30583 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9270/tid=9278] ppid=9268 vsize=341140 CPUtime=50.98 cores=0-7
/proc/9270/task/9278/stat : 9278 (plingeling) R 9268 9270 9244 0 -1 4202560 13244 1253 0 0 5096 2 0 0 25 0 9 0 702449411 349327360 30583 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9270/tid=9279] ppid=9268 vsize=341140 CPUtime=50.08 cores=0-7

################
# More data... #
################

/proc/9270/task/9280/stat : 9280 (plingeling) R 9268 9270 9244 0 -1 4202560 1880980 1253 0 0 471705 321 0 0 25 0 9 0 702449411 888209408 162114 33554432000 4194304 4417452 140736614271856 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9270/tid=9281] ppid=9268 vsize=867392 CPUtime=4722.31 cores=0-7
/proc/9270/task/9281/stat : 9281 (plingeling) R 9268 9270 9244 0 -1 4202560 1865218 1253 0 0 471903 328 0 0 25 0 9 0 702449411 888209408 162114 33554432000 4194304 4417452 140736614271856 18446744073709551615 4343986 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9270/tid=9282] ppid=9268 vsize=867392 CPUtime=4722.16 cores=0-7
/proc/9270/task/9282/stat : 9282 (plingeling) R 9268 9270 9244 0 -1 4202560 1976107 1253 0 0 471851 365 0 0 25 0 9 0 702449411 888209408 162114 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9270/tid=9283] ppid=9268 vsize=867392 CPUtime=4722.29 cores=0-7
/proc/9270/task/9283/stat : 9283 (plingeling) R 9268 9270 9244 0 -1 4202560 1806691 1253 0 0 471899 330 0 0 25 0 9 0 702449411 888209408 162114 33554432000 4194304 4417452 140736614271856 18446744073709551615 4232612 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9270/tid=9284] ppid=9268 vsize=867392 CPUtime=4722.31 cores=0-7
/proc/9270/task/9284/stat : 9284 (plingeling) R 9268 9270 9244 0 -1 4202560 1768133 1253 0 0 471904 327 0 0 25 0 9 0 702449412 888209408 162114 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364558 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 37692.1
Current children cumulated vsize (KiB) 867392

[startup+4782.3 s]
/proc/loadavg: 8.05 8.04 8.00 9/537 9423
/proc/meminfo: memFree=14817132/32951124 swapFree=51873512/67111528
[pid=9270] ppid=9268 vsize=929224 CPUtime=38170.9 cores=0-7
/proc/9270/stat : 9270 (plingeling) S 9268 9270 9244 0 -1 4202496 14810605 1253 0 0 3814432 2654 0 0 25 0 9 0 702449400 951525376 177572 33554432000 4194304 4417452 140736614271856 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9270/statm: 232306 177572 179 55 0 230192 0
[pid=9270/tid=9277] ppid=9268 vsize=929224 CPUtime=4782.27 cores=0-7
/proc/9270/task/9277/stat : 9277 (plingeling) R 9268 9270 9244 0 -1 4202560 1745100 1253 0 0 477925 302 0 0 25 0 9 0 702449411 951525376 177572 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9270/tid=9278] ppid=9268 vsize=929224 CPUtime=4782.16 cores=0-7
/proc/9270/task/9278/stat : 9278 (plingeling) R 9268 9270 9244 0 -1 4202560 1874119 1253 0 0 477891 325 0 0 25 0 9 0 702449411 951525376 177572 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9270/tid=9279] ppid=9268 vsize=929224 CPUtime=4697.07 cores=0-7
/proc/9270/task/9279/stat : 9279 (plingeling) R 9268 9270 9244 0 -1 4202560 1802482 1253 0 0 469373 334 0 0 25 0 9 0 702449411 951525376 177572 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9270/tid=9280] ppid=9268 vsize=929224 CPUtime=4780.22 cores=0-7
/proc/9270/task/9280/stat : 9280 (plingeling) R 9268 9270 9244 0 -1 4202560 1890686 1253 0 0 477700 322 0 0 25 0 9 0 702449411 951525376 177572 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9270/tid=9281] ppid=9268 vsize=929224 CPUtime=4782.29 cores=0-7
/proc/9270/task/9281/stat : 9281 (plingeling) R 9268 9270 9244 0 -1 4202560 1881784 1253 0 0 477898 331 0 0 25 0 9 0 702449411 951525376 177572 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9270/tid=9282] ppid=9268 vsize=929224 CPUtime=4782.14 cores=0-7
/proc/9270/task/9282/stat : 9282 (plingeling) R 9268 9270 9244 0 -1 4202560 1992817 1253 0 0 477845 369 0 0 25 0 9 0 702449411 951525376 177572 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9270/tid=9283] ppid=9268 vsize=929224 CPUtime=4782.27 cores=0-7
/proc/9270/task/9283/stat : 9283 (plingeling) R 9268 9270 9244 0 -1 4202560 1820693 1253 0 0 477894 333 0 0 25 0 9 0 702449411 951525376 177572 33554432000 4194304 4417452 140736614271856 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9270/tid=9284] ppid=9268 vsize=929224 CPUtime=4782.28 cores=0-7
/proc/9270/task/9284/stat : 9284 (plingeling) R 9268 9270 9244 0 -1 4202560 1797176 1253 0 0 477895 333 0 0 25 0 9 0 702449412 951525376 177572 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38170.9
Current children cumulated vsize (KiB) 929224

[startup+4842.3 s]
/proc/loadavg: 8.24 8.09 8.02 9/537 9425
/proc/meminfo: memFree=14835852/32951124 swapFree=51873512/67111528
[pid=9270] ppid=9268 vsize=910816 CPUtime=38649.8 cores=0-7
/proc/9270/stat : 9270 (plingeling) S 9268 9270 9244 0 -1 4202496 14973802 1253 0 0 3862293 2685 0 0 25 0 9 0 702449400 932675584 172970 33554432000 4194304 4417452 140736614271856 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9270/statm: 227704 172970 179 55 0 225590 0
[pid=9270/tid=9277] ppid=9268 vsize=910816 CPUtime=4842.27 cores=0-7
/proc/9270/task/9277/stat : 9277 (plingeling) R 9268 9270 9244 0 -1 4202560 1764919 1253 0 0 483922 305 0 0 25 0 9 0 702449411 932675584 172970 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9270/tid=9278] ppid=9268 vsize=910816 CPUtime=4842.17 cores=0-7
/proc/9270/task/9278/stat : 9278 (plingeling) R 9268 9270 9244 0 -1 4202560 1894555 1253 0 0 483887 330 0 0 25 0 9 0 702449411 932675584 172970 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9270/tid=9279] ppid=9268 vsize=910816 CPUtime=4755.98 cores=0-7
/proc/9270/task/9279/stat : 9279 (plingeling) R 9268 9270 9244 0 -1 4202560 1817611 1253 0 0 475260 338 0 0 25 0 9 0 702449411 932675584 172970 33554432000 4194304 4417452 140736614271856 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9270/tid=9280] ppid=9268 vsize=910816 CPUtime=4840.2 cores=0-7
/proc/9270/task/9280/stat : 9280 (plingeling) R 9268 9270 9244 0 -1 4202560 1916355 1253 0 0 483694 326 0 0 25 0 9 0 702449411 932675584 172970 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9270/tid=9281] ppid=9268 vsize=910816 CPUtime=4842.29 cores=0-7
/proc/9270/task/9281/stat : 9281 (plingeling) R 9268 9270 9244 0 -1 4202560 1911553 1253 0 0 483893 336 0 0 25 0 9 0 702449411 932675584 172970 33554432000 4194304 4417452 140736614271856 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9270/tid=9282] ppid=9268 vsize=910816 CPUtime=4842.15 cores=0-7
/proc/9270/task/9282/stat : 9282 (plingeling) R 9268 9270 9244 0 -1 4202560 2012317 1253 0 0 483842 373 0 0 25 0 9 0 702449411 932675584 172970 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9270/tid=9283] ppid=9268 vsize=910816 CPUtime=4842.27 cores=0-7
/proc/9270/task/9283/stat : 9283 (plingeling) R 9268 9270 9244 0 -1 4202560 1837324 1253 0 0 483891 336 0 0 25 0 9 0 702449411 932675584 172970 33554432000 4194304 4417452 140736614271856 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9270/tid=9284] ppid=9268 vsize=910816 CPUtime=4842.29 cores=0-7
/proc/9270/task/9284/stat : 9284 (plingeling) R 9268 9270 9244 0 -1 4202560 1813420 1253 0 0 483892 337 0 0 25 0 9 0 702449412 932675584 172970 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38649.8
Current children cumulated vsize (KiB) 910816

[startup+4902.3 s]
/proc/loadavg: 8.19 8.10 8.02 9/537 9427
/proc/meminfo: memFree=14858064/32951124 swapFree=51873512/67111528
[pid=9270] ppid=9268 vsize=891420 CPUtime=39128.7 cores=0-7
/proc/9270/stat : 9270 (plingeling) S 9268 9270 9244 0 -1 4202496 15119382 1253 0 0 3910156 2714 0 0 25 0 9 0 702449400 912814080 168121 33554432000 4194304 4417452 140736614271856 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9270/statm: 222855 168121 179 55 0 220741 0
[pid=9270/tid=9277] ppid=9268 vsize=891420 CPUtime=4902.27 cores=0-7
/proc/9270/task/9277/stat : 9277 (plingeling) R 9268 9270 9244 0 -1 4202560 1784721 1253 0 0 489918 309 0 0 25 0 9 0 702449411 912814080 168121 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9270/tid=9278] ppid=9268 vsize=891420 CPUtime=4902.17 cores=0-7
/proc/9270/task/9278/stat : 9278 (plingeling) R 9268 9270 9244 0 -1 4202560 1904254 1253 0 0 489886 331 0 0 25 0 9 0 702449411 912814080 168121 33554432000 4194304 4417452 140736614271856 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9270/tid=9279] ppid=9268 vsize=891420 CPUtime=4814.9 cores=0-7
/proc/9270/task/9279/stat : 9279 (plingeling) R 9268 9270 9244 0 -1 4202560 1833295 1253 0 0 481149 341 0 0 25 0 9 0 702449411 912814080 168121 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9270/tid=9280] ppid=9268 vsize=891420 CPUtime=4900.19 cores=0-7
/proc/9270/task/9280/stat : 9280 (plingeling) R 9268 9270 9244 0 -1 4202560 1939146 1253 0 0 489689 330 0 0 25 0 9 0 702449411 912814080 168121 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9270/tid=9281] ppid=9268 vsize=891420 CPUtime=4902.29 cores=0-7
/proc/9270/task/9281/stat : 9281 (plingeling) R 9268 9270 9244 0 -1 4202560 1924281 1253 0 0 489890 339 0 0 25 0 9 0 702449411 912814080 168121 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9270/tid=9282] ppid=9268 vsize=891420 CPUtime=4902.14 cores=0-7
/proc/9270/task/9282/stat : 9282 (plingeling) R 9268 9270 9244 0 -1 4202560 2031094 1253 0 0 489838 376 0 0 25 0 9 0 702449411 912814080 168121 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9270/tid=9283] ppid=9268 vsize=891420 CPUtime=4902.26 cores=0-7
/proc/9270/task/9283/stat : 9283 (plingeling) R 9268 9270 9244 0 -1 4202560 1862183 1253 0 0 489885 341 0 0 25 0 9 0 702449411 912814080 168121 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9270/tid=9284] ppid=9268 vsize=891420 CPUtime=4902.28 cores=0-7
/proc/9270/task/9284/stat : 9284 (plingeling) R 9268 9270 9244 0 -1 4202560 1834660 1253 0 0 489887 341 0 0 25 0 9 0 702449412 912814080 168121 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39128.7
Current children cumulated vsize (KiB) 891420

[startup+4962.3 s]
/proc/loadavg: 8.18 8.11 8.03 9/537 9437
/proc/meminfo: memFree=14804736/32951124 swapFree=51873512/67111528
[pid=9270] ppid=9268 vsize=868832 CPUtime=39607.6 cores=0-7
/proc/9270/stat : 9270 (plingeling) S 9268 9270 9244 0 -1 4202496 15264848 1253 0 0 3958013 2742 0 0 25 0 9 0 702449400 889683968 162474 33554432000 4194304 4417452 140736614271856 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9270/statm: 217208 162474 179 55 0 215094 0
[pid=9270/tid=9277] ppid=9268 vsize=868832 CPUtime=4962.25 cores=0-7
/proc/9270/task/9277/stat : 9277 (plingeling) R 9268 9270 9244 0 -1 4202560 1800878 1253 0 0 495913 312 0 0 25 0 9 0 702449411 889683968 162474 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9270/tid=9278] ppid=9268 vsize=868832 CPUtime=4962.16 cores=0-7
/proc/9270/task/9278/stat : 9278 (plingeling) R 9268 9270 9244 0 -1 4202560 1920550 1253 0 0 495882 334 0 0 25 0 9 0 702449411 889683968 162474 33554432000 4194304 4417452 140736614271856 18446744073709551615 4365973 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9270/tid=9279] ppid=9268 vsize=868832 CPUtime=4873.8 cores=0-7
/proc/9270/task/9279/stat : 9279 (plingeling) R 9268 9270 9244 0 -1 4202560 1848834 1253 0 0 487036 344 0 0 25 0 9 0 702449411 889683968 162474 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364812 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9270/tid=9280] ppid=9268 vsize=868832 CPUtime=4960.17 cores=0-7
/proc/9270/task/9280/stat : 9280 (plingeling) R 9268 9270 9244 0 -1 4202560 1956896 1253 0 0 495682 335 0 0 25 0 9 0 702449411 889683968 162474 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9270/tid=9281] ppid=9268 vsize=868832 CPUtime=4962.29 cores=0-7
/proc/9270/task/9281/stat : 9281 (plingeling) R 9268 9270 9244 0 -1 4202560 1939714 1253 0 0 495887 342 0 0 25 0 9 0 702449411 889683968 162474 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9270/tid=9282] ppid=9268 vsize=868832 CPUtime=4962.15 cores=0-7
/proc/9270/task/9282/stat : 9282 (plingeling) R 9268 9270 9244 0 -1 4202560 2051996 1253 0 0 495835 380 0 0 25 0 9 0 702449411 889683968 162474 33554432000 4194304 4417452 140736614271856 18446744073709551615 4365976 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9270/tid=9283] ppid=9268 vsize=868832 CPUtime=4962.27 cores=0-7
/proc/9270/task/9283/stat : 9283 (plingeling) R 9268 9270 9244 0 -1 4202560 1882666 1253 0 0 495882 345 0 0 25 0 9 0 702449411 889683968 162474 33554432000 4194304 4417452 140736614271856 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9270/tid=9284] ppid=9268 vsize=868832 CPUtime=4962.29 cores=0-7
/proc/9270/task/9284/stat : 9284 (plingeling) R 9268 9270 9244 0 -1 4202560 1857566 1253 0 0 495884 345 0 0 25 0 9 0 702449412 889683968 162474 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39607.6
Current children cumulated vsize (KiB) 868832



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5011.5 s]
/proc/loadavg: 8.08 8.09 8.02 10/537 9438
/proc/meminfo: memFree=14881268/32951124 swapFree=51873512/67111528
[pid=9270] ppid=9268 vsize=865148 CPUtime=40000.4 cores=0-7
/proc/9270/stat : 9270 (plingeling) S 9268 9270 9244 0 -1 4202496 15372832 1253 0 0 3997283 2762 0 0 25 0 9 0 702449400 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9270/statm: 216287 161553 179 55 0 214173 0
[pid=9270/tid=9277] ppid=9268 vsize=865148 CPUtime=5011.49 cores=0-7
/proc/9270/task/9277/stat : 9277 (plingeling) R 9268 9270 9244 0 -1 4202560 1814850 1253 0 0 500834 315 0 0 25 0 9 0 702449411 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9270/tid=9278] ppid=9268 vsize=865148 CPUtime=5011.4 cores=0-7
/proc/9270/task/9278/stat : 9278 (plingeling) R 9268 9270 9244 0 -1 4202560 1937982 1253 0 0 500803 337 0 0 25 0 9 0 702449411 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9270/tid=9279] ppid=9268 vsize=865148 CPUtime=4922.13 cores=0-7
/proc/9270/task/9279/stat : 9279 (plingeling) R 9268 9270 9244 0 -1 4202560 1858371 1253 0 0 491866 347 0 0 25 0 9 0 702449411 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9270/tid=9280] ppid=9268 vsize=865148 CPUtime=5009.39 cores=0-7
/proc/9270/task/9280/stat : 9280 (plingeling) R 9268 9270 9244 0 -1 4202560 1971140 1253 0 0 500602 337 0 0 25 0 9 0 702449411 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9270/tid=9281] ppid=9268 vsize=865148 CPUtime=5011.54 cores=0-7
/proc/9270/task/9281/stat : 9281 (plingeling) R 9268 9270 9244 0 -1 4202560 1950750 1253 0 0 500809 345 0 0 25 0 9 0 702449411 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364958 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9270/tid=9282] ppid=9268 vsize=865148 CPUtime=5011.39 cores=0-7
/proc/9270/task/9282/stat : 9282 (plingeling) R 9268 9270 9244 0 -1 4202560 2065994 1253 0 0 500757 382 0 0 25 0 9 0 702449411 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9270/tid=9283] ppid=9268 vsize=865148 CPUtime=5011.51 cores=0-7
/proc/9270/task/9283/stat : 9283 (plingeling) R 9268 9270 9244 0 -1 4202560 1899377 1253 0 0 500803 348 0 0 25 0 9 0 702449411 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 4366533 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9270/tid=9284] ppid=9268 vsize=865148 CPUtime=5011.53 cores=0-7
/proc/9270/task/9284/stat : 9284 (plingeling) R 9268 9270 9244 0 -1 4202560 1868620 1253 0 0 500806 347 0 0 25 0 9 0 702449412 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 865148

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5011.5 s]
/proc/loadavg: 8.08 8.09 8.02 10/537 9438
/proc/meminfo: memFree=14881268/32951124 swapFree=51873512/67111528
[pid=9270] ppid=9268 vsize=865148 CPUtime=40000.4 cores=0-7
/proc/9270/stat : 9270 (plingeling) S 9268 9270 9244 0 -1 4202496 15372832 1253 0 0 3997283 2762 0 0 25 0 9 0 702449400 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9270/statm: 216287 161553 179 55 0 214173 0
[pid=9270/tid=9277] ppid=9268 vsize=865148 CPUtime=5011.49 cores=0-7
/proc/9270/task/9277/stat : 9277 (plingeling) R 9268 9270 9244 0 -1 4202560 1814850 1253 0 0 500834 315 0 0 25 0 9 0 702449411 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9270/tid=9278] ppid=9268 vsize=865148 CPUtime=5011.4 cores=0-7
/proc/9270/task/9278/stat : 9278 (plingeling) R 9268 9270 9244 0 -1 4202560 1937982 1253 0 0 500803 337 0 0 25 0 9 0 702449411 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9270/tid=9279] ppid=9268 vsize=865148 CPUtime=4922.13 cores=0-7
/proc/9270/task/9279/stat : 9279 (plingeling) R 9268 9270 9244 0 -1 4202560 1858371 1253 0 0 491866 347 0 0 25 0 9 0 702449411 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9270/tid=9280] ppid=9268 vsize=865148 CPUtime=5009.39 cores=0-7
/proc/9270/task/9280/stat : 9280 (plingeling) R 9268 9270 9244 0 -1 4202560 1971140 1253 0 0 500602 337 0 0 25 0 9 0 702449411 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9270/tid=9281] ppid=9268 vsize=865148 CPUtime=5011.54 cores=0-7
/proc/9270/task/9281/stat : 9281 (plingeling) R 9268 9270 9244 0 -1 4202560 1950750 1253 0 0 500809 345 0 0 25 0 9 0 702449411 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364958 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9270/tid=9282] ppid=9268 vsize=865148 CPUtime=5011.39 cores=0-7
/proc/9270/task/9282/stat : 9282 (plingeling) R 9268 9270 9244 0 -1 4202560 2065994 1253 0 0 500757 382 0 0 25 0 9 0 702449411 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9270/tid=9283] ppid=9268 vsize=865148 CPUtime=5011.51 cores=0-7
/proc/9270/task/9283/stat : 9283 (plingeling) R 9268 9270 9244 0 -1 4202560 1899377 1253 0 0 500803 348 0 0 25 0 9 0 702449411 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 4366533 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9270/tid=9284] ppid=9268 vsize=865148 CPUtime=5011.53 cores=0-7
/proc/9270/task/9284/stat : 9284 (plingeling) R 9268 9270 9244 0 -1 4202560 1868620 1253 0 0 500806 347 0 0 25 0 9 0 702449412 885911552 161553 33554432000 4194304 4417452 140736614271856 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 865148

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5011.58
CPU time (s): 40000.6
CPU user time (s): 39973
CPU system time (s): 27.6598
CPU usage (%): 798.165
Max. virtual memory (cumulated for all children) (KiB): 982676

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39973
system time used= 27.6598
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 15374085
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 807
involuntary context switches= 186935

runsolver used 25.6631 second user time and 63.7433 second system time

The end

Launcher Data 

Begin job on node146 at 2011-05-22 23:38:51
IDJOB=3394614
IDBENCH=83321
IDSOLVER=1851
FILE ID=node146/3394614-1306100331
RUNJOBID= node146-1306100331-9258
PBS_JOBID= 13387986
Free space on /tmp= 73036 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/kullmann/VanDerWaerden/VanderWaerden3k/VanDerWaerden_2-3-20_389.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394614-1306100331/watcher-3394614-1306100331 -o /tmp/evaluation-result-3394614-1306100331/solver-3394614-1306100331 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394614-1306100331.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= f105570f47bdca662c2d3d185f8dc1f8
RANDOM SEED=1880097634

node146.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      15527416 kB
Buffers:        311352 kB
Cached:        4766188 kB
SwapCached:      56204 kB
Active:       12506008 kB
Inactive:      4602104 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      15527416 kB
SwapTotal:    67111528 kB
SwapFree:     51873512 kB
Dirty:            1176 kB
Writeback:           0 kB
AnonPages:    11981864 kB
Mapped:          29572 kB
Slab:           144124 kB
PageTables:      98064 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 160669088 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73032 MiB
End job on node146 at 2011-05-23 01:02:25