Trace number 3372971

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)UNSAT 9762.18 1243.11

General information on the benchmark

NameSAT-Race-2010-CNF/hardware-verification/
velev/1dlx_c_iq60_a.cnf
MD5SUM1e60a63f1ba68b9b3c423ca413b4fa2b
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark718.86
Satisfiable
(Un)Satisfiability was proved
Number of variables651875
Number of clauses9927770
Sum of the clauses size28821360
Maximum clause length195
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 28783867
Number of clauses of size 3839934
Number of clauses of size 410777
Number of clauses of size 570916
Number of clauses of size over 5222275

Solver Data 

9762.15/1243.10	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372971-1305445414/watcher-3372971-1305445414 -o /tmp/evaluation-result-3372971-1305445414/solver-3372971-1305445414 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3372971-1305445414.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.70 7.83 7.64 2/357 25796
/proc/meminfo: memFree=22457480/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=16804 CPUtime=0 cores=0-7
/proc/25796/stat : 25796 (plingeling) R 25794 25796 25304 0 -1 4202496 2282 1253 0 0 0 0 0 0 23 0 1 0 734772196 17207296 2212 33554432000 4194304 4417452 140733542182848 18446744073709551615 260905393920 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/25796/statm: 4201 2212 148 55 0 2087 0

[startup+0.0297369 s]
/proc/loadavg: 7.70 7.83 7.64 2/357 25796
/proc/meminfo: memFree=22457480/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=20428 CPUtime=0.02 cores=0-7
/proc/25796/stat : 25796 (plingeling) R 25794 25796 25304 0 -1 4202496 3185 1253 0 0 2 0 0 0 23 0 1 0 734772196 20918272 3115 33554432000 4194304 4417452 140733542182848 18446744073709551615 4259665 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/25796/statm: 5107 3115 149 55 0 2993 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 20428

[startup+0.100725 s]
/proc/loadavg: 7.70 7.83 7.64 2/357 25796
/proc/meminfo: memFree=22457480/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=31848 CPUtime=0.08 cores=0-7
/proc/25796/stat : 25796 (plingeling) R 25794 25796 25304 0 -1 4202496 6056 1253 0 0 8 0 0 0 24 0 1 0 734772196 32612352 5986 33554432000 4194304 4417452 140733542182848 18446744073709551615 4262065 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/25796/statm: 7962 5986 149 55 0 5848 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 31848

[startup+0.300688 s]
/proc/loadavg: 7.70 7.83 7.64 2/357 25796
/proc/meminfo: memFree=22457480/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=50820 CPUtime=0.28 cores=0-7
/proc/25796/stat : 25796 (plingeling) R 25794 25796 25304 0 -1 4202496 10831 1253 0 0 27 1 0 0 25 0 1 0 734772196 52039680 10729 33554432000 4194304 4417452 140733542182848 18446744073709551615 4227899 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/25796/statm: 12705 10729 149 55 0 10591 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 50820

[startup+0.700615 s]
/proc/loadavg: 7.70 7.83 7.64 2/357 25796
/proc/meminfo: memFree=22457480/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=121756 CPUtime=0.69 cores=0-7
/proc/25796/stat : 25796 (plingeling) R 25794 25796 25304 0 -1 4202496 28567 1253 0 0 65 4 0 0 25 0 1 0 734772196 124678144 28463 33554432000 4194304 4417452 140733542182848 18446744073709551615 260905451946 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/25796/statm: 30695 28469 149 55 0 28581 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 121756

[startup+1.50047 s]
/proc/loadavg: 7.70 7.83 7.64 2/358 25802
/proc/meminfo: memFree=22328428/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=218552 CPUtime=1.49 cores=0-7
/proc/25796/stat : 25796 (plingeling) R 25794 25796 25304 0 -1 4202496 52766 1253 0 0 142 7 0 0 25 0 1 0 734772196 223797248 52662 33554432000 4194304 4417452 140733542182848 18446744073709551615 260905393920 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/25796/statm: 54638 52662 149 55 0 52524 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 218552

[startup+3.10018 s]
/proc/loadavg: 7.70 7.83 7.64 2/358 25802
/proc/meminfo: memFree=22208396/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=420144 CPUtime=3.08 cores=0-7
/proc/25796/stat : 25796 (plingeling) R 25794 25796 25304 0 -1 4202496 103202 1253 0 0 295 13 0 0 25 0 1 0 734772196 430227456 103060 33554432000 4194304 4417452 140733542182848 18446744073709551615 260905393920 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/25796/statm: 105036 103060 149 55 0 102922 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 420144

[startup+6.30066 s]
/proc/loadavg: 7.16 7.71 7.60 2/358 25802
/proc/meminfo: memFree=21911412/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=822072 CPUtime=6.29 cores=0-7
/proc/25796/stat : 25796 (plingeling) R 25794 25796 25304 0 -1 4202496 203684 1253 0 0 604 25 0 0 25 0 1 0 734772196 841801728 203542 33554432000 4194304 4417452 140733542182848 18446744073709551615 260905393920 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/25796/statm: 205518 203542 149 55 0 203404 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 822072

[startup+12.7005 s]
/proc/loadavg: 6.67 7.60 7.57 2/358 25803
/proc/meminfo: memFree=20842084/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=1627076 CPUtime=12.69 cores=0-7
/proc/25796/stat : 25796 (plingeling) R 25794 25796 25304 0 -1 4202496 405032 1253 0 0 1224 45 0 0 25 0 1 0 734772196 1666125824 404793 33554432000 4194304 4417452 140733542182848 18446744073709551615 4231265 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/25796/statm: 406769 404793 149 55 0 404655 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 1627076

[startup+25.5011 s]
/proc/loadavg: 5.41 7.28 7.46 9/366 25811
/proc/meminfo: memFree=19627924/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=3218572 CPUtime=36.93 cores=0-7
/proc/25796/stat : 25796 (plingeling) S 25794 25796 25304 0 -1 4202496 945398 1253 0 0 3572 121 0 0 25 0 9 0 734772196 3295817728 769944 33554432000 4194304 4417452 140733542182848 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25796/statm: 804643 769944 168 55 0 802529 0
[pid=25796/tid=25804] ppid=25794 vsize=3218572 CPUtime=1.63 cores=0-7
/proc/25796/task/25804/stat : 25804 (plingeling) R 25794 25796 25304 0 -1 4202560 17995 1253 0 0 160 3 0 0 20 0 9 0 734774581 3295817728 769944 33554432000 4194304 4417452 140733542182848 18446744073709551615 4284980 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25796/tid=25805] ppid=25794 vsize=3218572 CPUtime=1.63 cores=0-7
/proc/25796/task/25805/stat : 25805 (plingeling) R 25794 25796 25304 0 -1 4202560 18035 1253 0 0 160 3 0 0 25 0 9 0 734774581 3295817728 769944 33554432000 4194304 4417452 140733542182848 18446744073709551615 4287195 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=25796/tid=25806] ppid=25794 vsize=3218572 CPUtime=1.64 cores=0-7
/proc/25796/task/25806/stat : 25806 (plingeling) R 25794 25796 25304 0 -1 4202560 18014 1253 0 0 161 3 0 0 25 0 9 0 734774581 3295817728 769944 33554432000 4194304 4417452 140733542182848 18446744073709551615 4288818 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=25796/tid=25807] ppid=25794 vsize=3218572 CPUtime=1.64 cores=0-7
/proc/25796/task/25807/stat : 25807 (plingeling) R 25794 25796 25304 0 -1 4202560 11645 1253 0 0 161 3 0 0 25 0 9 0 734774581 3295817728 769944 33554432000 4194304 4417452 140733542182848 18446744073709551615 4232437 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=25796/tid=25808] ppid=25794 vsize=3218572 CPUtime=1.59 cores=0-7
/proc/25796/task/25808/stat : 25808 (plingeling) R 25794 25796 25304 0 -1 4202560 17984 1253 0 0 157 2 0 0 25 0 9 0 734774581 3295817728 769944 33554432000 4194304 4417452 140733542182848 18446744073709551615 4287187 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25796/tid=25809] ppid=25794 vsize=3218572 CPUtime=1.63 cores=0-7
/proc/25796/task/25809/stat : 25809 (plingeling) R 25794 25796 25304 0 -1 4202560 17989 1253 0 0 159 4 0 0 25 0 9 0 734774582 3295817728 769944 33554432000 4194304 4417452 140733542182848 18446744073709551615 4288226 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25796/tid=25810] ppid=25794 vsize=3218572 CPUtime=1.62 cores=0-7
/proc/25796/task/25810/stat : 25810 (plingeling) R 25794 25796 25304 0 -1 4202560 17999 1253 0 0 159 3 0 0 25 0 9 0 734774582 3295817728 769944 33554432000 4194304 4417452 140733542182848 18446744073709551615 4285094 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25796/tid=25811] ppid=25794 vsize=3218572 CPUtime=1.63 cores=0-7
/proc/25796/task/25811/stat : 25811 (plingeling) R 25794 25796 25304 0 -1 4202560 17998 1253 0 0 160 3 0 0 25 0 9 0 734774582 3295817728 769944 33554432000 4194304 4417452 140733542182848 18446744073709551615 4240353 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 36.93
Current children cumulated vsize (KiB) 3218572

[startup+51.1004 s]
/proc/loadavg: 6.23 7.32 7.47 9/366 25812
/proc/meminfo: memFree=19358296/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=3238792 CPUtime=241.07 cores=0-7
/proc/25796/stat : 25796 (plingeling) S 25794 25796 25304 0 -1 4202496 1558939 1253 0 0 23871 236 0 0 25 0 9 0 734772196 3316523008 775105 33554432000 4194304 4417452 140733542182848 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25796/statm: 809698 775105 180 55 0 807584 0
[pid=25796/tid=25804] ppid=25794 vsize=3238792 CPUtime=26.94 cores=0-7
/proc/25796/task/25804/stat : 25804 (plingeling) R 25794 25796 25304 0 -1 4202560 126906 1253 0 0 2669 25 0 0 25 0 9 0 734774581 3316523008 775105 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25796/tid=25805] ppid=25794 vsize=3238792 CPUtime=27.21 cores=0-7
/proc/25796/task/25805/stat : 25805 (plingeling) R 25794 25796 25304 0 -1 4202560 85683 1253 0 0 2707 14 0 0 25 0 9 0 734774581 3316523008 775105 33554432000 4194304 4417452 140733542182848 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=25796/tid=25806] ppid=25794 vsize=3238792 CPUtime=27.22 cores=0-7
/proc/25796/task/25806/stat : 25806 (plingeling) R 25794 25796 25304 0 -1 4202560 90347 1253 0 0 2704 18 0 0 25 0 9 0 734774581 3316523008 775105 33554432000 4194304 4417452 140733542182848 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25796/tid=25807] ppid=25794 vsize=3238792 CPUtime=27.23 cores=0-7
/proc/25796/task/25807/stat : 25807 (plingeling) R 25794 25796 25304 0 -1 4202560 16001 1253 0 0 2720 3 0 0 25 0 9 0 734774581 3316523008 775105 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236616 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=25796/tid=25808] ppid=25794 vsize=3238792 CPUtime=27 cores=0-7
/proc/25796/task/25808/stat : 25808 (plingeling) R 25794 25796 25304 0 -1 4202560 126193 1253 0 0 2671 29 0 0 25 0 9 0 734774581 3316523008 775105 33554432000 4194304 4417452 140733542182848 18446744073709551615 4232584 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=25796/tid=25809] ppid=25794 vsize=3238792 CPUtime=27.19 cores=0-7
/proc/25796/task/25809/stat : 25809 (plingeling) R 25794 25796 25304 0 -1 4202560 100332 1253 0 0 2699 20 0 0 25 0 9 0 734774582 3316523008 775105 33554432000 4194304 4417452 140733542182848 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25796/tid=25810] ppid=25794 vsize=3238792 CPUtime=27.19 cores=0-7
/proc/25796/task/25810/stat : 25810 (plingeling) R 25794 25796 25304 0 -1 4202560 96993 1253 0 0 2706 13 0 0 25 0 9 0 734774582 3316523008 775105 33554432000 4194304 4417452 140733542182848 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25796/tid=25811] ppid=25794 vsize=3238792 CPUtime=27.17 cores=0-7
/proc/25796/task/25811/stat : 25811 (plingeling) R 25794 25796 25304 0 -1 4202560 108745 1253 0 0 2700 17 0 0 25 0 9 0 734774582 3316523008 775105 33554432000 4194304 4417452 140733542182848 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 241.07
Current children cumulated vsize (KiB) 3238792

[startup+102.314 s]
/proc/loadavg: 7.28 7.44 7.50 9/366 25813
/proc/meminfo: memFree=19326308/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=3271560 CPUtime=650.18 cores=0-7
/proc/25796/stat : 25796 (plingeling) S 25794 25796 25304 0 -1 4202496 1567131 1253 0 0 64780 238 0 0 25 0 9 0 734772196 3350077440 783297 33554432000 4194304 4417452 140733542182848 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25796/statm: 817890 783297 180 55 0 815776 0
[pid=25796/tid=25804] ppid=25794 vsize=3271560 CPUtime=77.54 cores=0-7
/proc/25796/task/25804/stat : 25804 (plingeling) R 25794 25796 25304 0 -1 4202560 131002 1253 0 0 7729 25 0 0 25 0 9 0 734774581 3350077440 783297 33554432000 4194304 4417452 140733542182848 18446744073709551615 4265465 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25796/tid=25805] ppid=25794 vsize=3271560 CPUtime=78.43 cores=0-7
/proc/25796/task/25805/stat : 25805 (plingeling) R 25794 25796 25304 0 -1 4202560 85683 1253 0 0 7829 14 0 0 25 0 9 0 734774581 3350077440 783297 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=25796/tid=25806] ppid=25794 vsize=3271560 CPUtime=78.44 cores=0-7
/proc/25796/task/25806/stat : 25806 (plingeling) R 25794 25796 25304 0 -1 4202560 90347 1253 0 0 7826 18 0 0 25 0 9 0 734774581 3350077440 783297 33554432000 4194304 4417452 140733542182848 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25796/tid=25807] ppid=25794 vsize=3271560 CPUtime=78.45 cores=0-7
/proc/25796/task/25807/stat : 25807 (plingeling) R 25794 25796 25304 0 -1 4202560 16001 1253 0 0 7842 3 0 0 25 0 9 0 734774581 3350077440 783297 33554432000 4194304 4417452 140733542182848 18446744073709551615 4245664 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=25796/tid=25808] ppid=25794 vsize=3271560 CPUtime=78.21 cores=0-7
/proc/25796/task/25808/stat : 25808 (plingeling) R 25794 25796 25304 0 -1 4202560 130289 1253 0 0 7792 29 0 0 25 0 9 0 734774581 3350077440 783297 33554432000 4194304 4417452 140733542182848 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=25796/tid=25809] ppid=25794 vsize=3271560 CPUtime=78.4 cores=0-7
/proc/25796/task/25809/stat : 25809 (plingeling) R 25794 25796 25304 0 -1 4202560 100332 1253 0 0 7820 20 0 0 25 0 9 0 734774582 3350077440 783297 33554432000 4194304 4417452 140733542182848 18446744073709551615 4265465 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25796/tid=25810] ppid=25794 vsize=3271560 CPUtime=78.41 cores=0-7
/proc/25796/task/25810/stat : 25810 (plingeling) R 25794 25796 25304 0 -1 4202560 96993 1253 0 0 7828 13 0 0 25 0 9 0 734774582 3350077440 783297 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25796/tid=25811] ppid=25794 vsize=3271560 CPUtime=78.39 cores=0-7
/proc/25796/task/25811/stat : 25811 (plingeling) R 25794 25796 25304 0 -1 4202560 108745 1253 0 0 7822 17 0 0 25 0 9 0 734774582 3350077440 783297 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236603 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 650.18
Current children cumulated vsize (KiB) 3271560

[startup+162.301 s]
/proc/loadavg: 7.78 7.55 7.53 9/366 25815
/proc/meminfo: memFree=19301696/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=3322640 CPUtime=1129.36 cores=0-7
/proc/25796/stat : 25796 (plingeling) S 25794 25796 25304 0 -1 4202496 1579901 1253 0 0 112697 239 0 0 25 0 9 0 734772196 3402383360 789274 33554432000 4194304 4417452 140733542182848 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25796/statm: 830660 789274 180 55 0 828546 0
[pid=25796/tid=25804] ppid=25794 vsize=3322640 CPUtime=136.83 cores=0-7
/proc/25796/task/25804/stat : 25804 (plingeling) R 25794 25796 25304 0 -1 4202560 131002 1253 0 0 13658 25 0 0 25 0 9 0 734774581 3402383360 789274 33554432000 4194304 4417452 140733542182848 18446744073709551615 4237770 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25796/tid=25805] ppid=25794 vsize=3322640 CPUtime=138.41 cores=0-7
/proc/25796/task/25805/stat : 25805 (plingeling) R 25794 25796 25304 0 -1 4202560 85683 1253 0 0 13827 14 0 0 25 0 9 0 734774581 3402383360 789274 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=25796/tid=25806] ppid=25794 vsize=3322640 CPUtime=138.42 cores=0-7
/proc/25796/task/25806/stat : 25806 (plingeling) R 25794 25796 25304 0 -1 4202560 94926 1253 0 0 13824 18 0 0 25 0 9 0 734774581 3402383360 789274 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236675 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25796/tid=25807] ppid=25794 vsize=3322640 CPUtime=138.43 cores=0-7
/proc/25796/task/25807/stat : 25807 (plingeling) R 25794 25796 25304 0 -1 4202560 16001 1253 0 0 13840 3 0 0 25 0 9 0 734774581 3402383360 789274 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=25796/tid=25808] ppid=25794 vsize=3322640 CPUtime=138.2 cores=0-7
/proc/25796/task/25808/stat : 25808 (plingeling) R 25794 25796 25304 0 -1 4202560 130289 1253 0 0 13791 29 0 0 25 0 9 0 734774581 3402383360 789274 33554432000 4194304 4417452 140733542182848 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=25796/tid=25809] ppid=25794 vsize=3322640 CPUtime=138.36 cores=0-7
/proc/25796/task/25809/stat : 25809 (plingeling) R 25794 25796 25304 0 -1 4202560 104437 1253 0 0 13816 20 0 0 25 0 9 0 734774582 3402383360 789274 33554432000 4194304 4417452 140733542182848 18446744073709551615 4265465 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25796/tid=25810] ppid=25794 vsize=3322640 CPUtime=138.4 cores=0-7
/proc/25796/task/25810/stat : 25810 (plingeling) R 25794 25796 25304 0 -1 4202560 96993 1253 0 0 13827 13 0 0 25 0 9 0 734774582 3402383360 789274 33554432000 4194304 4417452 140733542182848 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25796/tid=25811] ppid=25794 vsize=3322640 CPUtime=138.38 cores=0-7
/proc/25796/task/25811/stat : 25811 (plingeling) R 25794 25796 25304 0 -1 4202560 112831 1253 0 0 13820 18 0 0 25 0 9 0 734774582 3402383360 789274 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1129.36
Current children cumulated vsize (KiB) 3322640


################
# More data... #
################

/proc/25796/task/25808/stat : 25808 (plingeling) R 25794 25796 25304 0 -1 4202560 264656 1253 0 0 91775 47 0 0 25 0 9 0 734774581 2788499456 639444 33554432000 4194304 4417452 140733542182848 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=25796/tid=25809] ppid=25794 vsize=2723144 CPUtime=918.24 cores=0-7
/proc/25796/task/25809/stat : 25809 (plingeling) R 25794 25796 25304 0 -1 4202560 223906 1253 0 0 91788 36 0 0 25 0 9 0 734774582 2788499456 639444 33554432000 4194304 4417452 140733542182848 18446744073709551615 4254855 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25796/tid=25810] ppid=25794 vsize=2723144 CPUtime=918.42 cores=0-7
/proc/25796/task/25810/stat : 25810 (plingeling) R 25794 25796 25304 0 -1 4202560 223907 1253 0 0 91810 32 0 0 25 0 9 0 734774582 2788499456 639444 33554432000 4194304 4417452 140733542182848 18446744073709551615 4281694 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25796/tid=25811] ppid=25794 vsize=2723144 CPUtime=918.38 cores=0-7
/proc/25796/task/25811/stat : 25811 (plingeling) R 25794 25796 25304 0 -1 4202560 259335 1253 0 0 91802 36 0 0 25 0 9 0 734774582 2788499456 639444 33554432000 4194304 4417452 140733542182848 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 7360.1
Current children cumulated vsize (KiB) 2723144

[startup+1002.3 s]
/proc/loadavg: 8.09 8.04 7.83 9/366 25837
/proc/meminfo: memFree=19900004/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=2722732 CPUtime=7839.39 cores=0-7
/proc/25796/stat : 25796 (plingeling) S 25794 25796 25304 0 -1 4202496 2498174 1253 0 0 783575 364 0 0 25 0 9 0 734772196 2788077568 639341 33554432000 4194304 4417452 140733542182848 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25796/statm: 680683 639341 180 55 0 678569 0
[pid=25796/tid=25804] ppid=25794 vsize=2722732 CPUtime=966.92 cores=0-7
/proc/25796/task/25804/stat : 25804 (plingeling) R 25794 25796 25304 0 -1 4202560 230973 1253 0 0 96653 39 0 0 25 0 9 0 734774581 2788077568 639341 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25796/tid=25805] ppid=25794 vsize=2722732 CPUtime=978.4 cores=0-7
/proc/25796/task/25805/stat : 25805 (plingeling) R 25794 25796 25304 0 -1 4202560 228659 1253 0 0 97806 34 0 0 25 0 9 0 734774581 2788077568 639341 33554432000 4194304 4417452 140733542182848 18446744073709551615 4199406 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=25796/tid=25806] ppid=25794 vsize=2722732 CPUtime=978.45 cores=0-7
/proc/25796/task/25806/stat : 25806 (plingeling) R 25794 25796 25304 0 -1 4202560 226330 1253 0 0 97808 37 0 0 25 0 9 0 734774581 2788077568 639341 33554432000 4194304 4417452 140733542182848 18446744073709551615 4259140 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25796/tid=25807] ppid=25794 vsize=2722732 CPUtime=978.44 cores=0-7
/proc/25796/task/25807/stat : 25807 (plingeling) R 25794 25796 25304 0 -1 4202560 16001 1253 0 0 97841 3 0 0 25 0 9 0 734774581 2788077568 639341 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=25796/tid=25808] ppid=25794 vsize=2722732 CPUtime=978.22 cores=0-7
/proc/25796/task/25808/stat : 25808 (plingeling) R 25794 25796 25304 0 -1 4202560 264656 1253 0 0 97775 47 0 0 25 0 9 0 734774581 2788077568 639341 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=25796/tid=25809] ppid=25794 vsize=2722732 CPUtime=978.24 cores=0-7
/proc/25796/task/25809/stat : 25809 (plingeling) R 25794 25796 25304 0 -1 4202560 240292 1253 0 0 97785 39 0 0 25 0 9 0 734774582 2788077568 639341 33554432000 4194304 4417452 140733542182848 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25796/tid=25810] ppid=25794 vsize=2722732 CPUtime=978.42 cores=0-7
/proc/25796/task/25810/stat : 25810 (plingeling) R 25794 25796 25304 0 -1 4202560 223907 1253 0 0 97810 32 0 0 25 0 9 0 734774582 2788077568 639341 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25796/tid=25811] ppid=25794 vsize=2722732 CPUtime=978.38 cores=0-7
/proc/25796/task/25811/stat : 25811 (plingeling) R 25794 25796 25304 0 -1 4202560 259617 1253 0 0 97802 36 0 0 25 0 9 0 734774582 2788077568 639341 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 7839.39
Current children cumulated vsize (KiB) 2722732

[startup+1062.3 s]
/proc/loadavg: 8.03 8.03 7.83 9/366 25848
/proc/meminfo: memFree=19888160/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=2735072 CPUtime=8318.64 cores=0-7
/proc/25796/stat : 25796 (plingeling) S 25794 25796 25304 0 -1 4202496 2551582 1253 0 0 831492 372 0 0 25 0 9 0 734772196 2800713728 642394 33554432000 4194304 4417452 140733542182848 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25796/statm: 683768 642394 180 55 0 681654 0
[pid=25796/tid=25804] ppid=25794 vsize=2735072 CPUtime=1026.22 cores=0-7
/proc/25796/task/25804/stat : 25804 (plingeling) R 25794 25796 25304 0 -1 4202560 231249 1253 0 0 102583 39 0 0 25 0 9 0 734774581 2800713728 642394 33554432000 4194304 4417452 140733542182848 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25796/tid=25805] ppid=25794 vsize=2735072 CPUtime=1038.4 cores=0-7
/proc/25796/task/25805/stat : 25805 (plingeling) R 25794 25796 25304 0 -1 4202560 228659 1253 0 0 103806 34 0 0 25 0 9 0 734774581 2800713728 642394 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=25796/tid=25806] ppid=25794 vsize=2735072 CPUtime=1038.45 cores=0-7
/proc/25796/task/25806/stat : 25806 (plingeling) R 25794 25796 25304 0 -1 4202560 226330 1253 0 0 103808 37 0 0 25 0 9 0 734774581 2800713728 642394 33554432000 4194304 4417452 140733542182848 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25796/tid=25807] ppid=25794 vsize=2735072 CPUtime=1038.45 cores=0-7
/proc/25796/task/25807/stat : 25807 (plingeling) R 25794 25796 25304 0 -1 4202560 16001 1253 0 0 103842 3 0 0 25 0 9 0 734774581 2800713728 642394 33554432000 4194304 4417452 140733542182848 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=25796/tid=25808] ppid=25794 vsize=2735072 CPUtime=1038.21 cores=0-7
/proc/25796/task/25808/stat : 25808 (plingeling) R 25794 25796 25304 0 -1 4202560 281321 1253 0 0 103771 50 0 0 25 0 9 0 734774581 2800713728 642394 33554432000 4194304 4417452 140733542182848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=25796/tid=25809] ppid=25794 vsize=2735072 CPUtime=1038.22 cores=0-7
/proc/25796/task/25809/stat : 25809 (plingeling) R 25794 25796 25304 0 -1 4202560 240292 1253 0 0 103783 39 0 0 25 0 9 0 734774582 2800713728 642394 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25796/tid=25810] ppid=25794 vsize=2735072 CPUtime=1038.42 cores=0-7
/proc/25796/task/25810/stat : 25810 (plingeling) R 25794 25796 25304 0 -1 4202560 240572 1253 0 0 103808 34 0 0 25 0 9 0 734774582 2800713728 642394 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25796/tid=25811] ppid=25794 vsize=2735072 CPUtime=1038.39 cores=0-7
/proc/25796/task/25811/stat : 25811 (plingeling) R 25794 25796 25304 0 -1 4202560 279419 1253 0 0 103800 39 0 0 25 0 9 0 734774582 2800713728 642394 33554432000 4194304 4417452 140733542182848 18446744073709551615 4311324 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 8318.64
Current children cumulated vsize (KiB) 2735072

[startup+1122.3 s]
/proc/loadavg: 8.05 8.03 7.84 9/366 25849
/proc/meminfo: memFree=19803768/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=2819952 CPUtime=8797.93 cores=0-7
/proc/25796/stat : 25796 (plingeling) S 25794 25796 25304 0 -1 4202496 2641585 1253 0 0 879410 383 0 0 25 0 9 0 734772196 2887630848 663614 33554432000 4194304 4417452 140733542182848 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25796/statm: 704988 663614 180 55 0 702874 0
[pid=25796/tid=25804] ppid=25794 vsize=2819952 CPUtime=1085.5 cores=0-7
/proc/25796/task/25804/stat : 25804 (plingeling) R 25794 25796 25304 0 -1 4202560 252718 1253 0 0 108508 42 0 0 25 0 9 0 734774581 2887630848 663614 33554432000 4194304 4417452 140733542182848 18446744073709551615 4281657 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25796/tid=25805] ppid=25794 vsize=2819952 CPUtime=1098.4 cores=0-7
/proc/25796/task/25805/stat : 25805 (plingeling) R 25794 25796 25304 0 -1 4202560 247517 1253 0 0 109803 37 0 0 25 0 9 0 734774581 2887630848 663614 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=25796/tid=25806] ppid=25794 vsize=2819952 CPUtime=1098.45 cores=0-7
/proc/25796/task/25806/stat : 25806 (plingeling) R 25794 25796 25304 0 -1 4202560 252808 1253 0 0 109805 40 0 0 25 0 9 0 734774581 2887630848 663614 33554432000 4194304 4417452 140733542182848 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25796/tid=25807] ppid=25794 vsize=2819952 CPUtime=1098.45 cores=0-7
/proc/25796/task/25807/stat : 25807 (plingeling) R 25794 25796 25304 0 -1 4202560 16001 1253 0 0 109842 3 0 0 25 0 9 0 734774581 2887630848 663614 33554432000 4194304 4417452 140733542182848 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=25796/tid=25808] ppid=25794 vsize=2819952 CPUtime=1098.21 cores=0-7
/proc/25796/task/25808/stat : 25808 (plingeling) R 25794 25796 25304 0 -1 4202560 290087 1253 0 0 109770 51 0 0 25 0 9 0 734774581 2887630848 663614 33554432000 4194304 4417452 140733542182848 18446744073709551615 4354506 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=25796/tid=25809] ppid=25794 vsize=2819952 CPUtime=1098.21 cores=0-7
/proc/25796/task/25809/stat : 25809 (plingeling) R 25794 25796 25304 0 -1 4202560 241237 1253 0 0 109782 39 0 0 25 0 9 0 734774582 2887630848 663614 33554432000 4194304 4417452 140733542182848 18446744073709551615 4264940 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25796/tid=25810] ppid=25794 vsize=2819952 CPUtime=1098.42 cores=0-7
/proc/25796/task/25810/stat : 25810 (plingeling) R 25794 25796 25304 0 -1 4202560 245821 1253 0 0 109808 34 0 0 25 0 9 0 734774582 2887630848 663614 33554432000 4194304 4417452 140733542182848 18446744073709551615 4232584 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25796/tid=25811] ppid=25794 vsize=2819952 CPUtime=1098.39 cores=0-7
/proc/25796/task/25811/stat : 25811 (plingeling) R 25794 25796 25304 0 -1 4202560 287657 1253 0 0 109799 40 0 0 25 0 9 0 734774582 2887630848 663614 33554432000 4194304 4417452 140733542182848 18446744073709551615 4265465 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 8797.93
Current children cumulated vsize (KiB) 2819952

[startup+1182.3 s]
/proc/loadavg: 8.06 8.04 7.86 9/366 25851
/proc/meminfo: memFree=19807928/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=2817704 CPUtime=9277.23 cores=0-7
/proc/25796/stat : 25796 (plingeling) S 25794 25796 25304 0 -1 4202496 2674128 1253 0 0 927336 387 0 0 25 0 9 0 734772196 2885328896 663052 33554432000 4194304 4417452 140733542182848 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25796/statm: 704426 663052 180 55 0 702312 0
[pid=25796/tid=25804] ppid=25794 vsize=2817704 CPUtime=1144.8 cores=0-7
/proc/25796/task/25804/stat : 25804 (plingeling) R 25794 25796 25304 0 -1 4202560 257999 1253 0 0 114438 42 0 0 25 0 9 0 734774581 2885328896 663052 33554432000 4194304 4417452 140733542182848 18446744073709551615 4263096 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25796/tid=25805] ppid=25794 vsize=2817704 CPUtime=1158.4 cores=0-7
/proc/25796/task/25805/stat : 25805 (plingeling) R 25794 25796 25304 0 -1 4202560 252687 1253 0 0 115803 37 0 0 25 0 9 0 734774581 2885328896 663052 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236593 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=25796/tid=25806] ppid=25794 vsize=2817704 CPUtime=1158.44 cores=0-7
/proc/25796/task/25806/stat : 25806 (plingeling) R 25794 25796 25304 0 -1 4202560 257980 1253 0 0 115804 40 0 0 25 0 9 0 734774581 2885328896 663052 33554432000 4194304 4417452 140733542182848 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25796/tid=25807] ppid=25794 vsize=2817704 CPUtime=1158.45 cores=0-7
/proc/25796/task/25807/stat : 25807 (plingeling) R 25794 25796 25304 0 -1 4202560 16001 1253 0 0 115842 3 0 0 25 0 9 0 734774581 2885328896 663052 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=25796/tid=25808] ppid=25794 vsize=2817704 CPUtime=1158.21 cores=0-7
/proc/25796/task/25808/stat : 25808 (plingeling) R 25794 25796 25304 0 -1 4202560 290087 1253 0 0 115770 51 0 0 25 0 9 0 734774581 2885328896 663052 33554432000 4194304 4417452 140733542182848 18446744073709551615 4281853 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=25796/tid=25809] ppid=25794 vsize=2817704 CPUtime=1158.2 cores=0-7
/proc/25796/task/25809/stat : 25809 (plingeling) R 25794 25796 25304 0 -1 4202560 257894 1253 0 0 115779 41 0 0 25 0 9 0 734774582 2885328896 663052 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25796/tid=25810] ppid=25794 vsize=2817704 CPUtime=1158.42 cores=0-7
/proc/25796/task/25810/stat : 25810 (plingeling) R 25794 25796 25304 0 -1 4202560 245821 1253 0 0 115808 34 0 0 25 0 9 0 734774582 2885328896 663052 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25796/tid=25811] ppid=25794 vsize=2817704 CPUtime=1158.39 cores=0-7
/proc/25796/task/25811/stat : 25811 (plingeling) R 25794 25796 25304 0 -1 4202560 287920 1253 0 0 115799 40 0 0 25 0 9 0 734774582 2885328896 663052 33554432000 4194304 4417452 140733542182848 18446744073709551615 4358082 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 9277.23
Current children cumulated vsize (KiB) 2817704

[startup+1242.3 s]
/proc/loadavg: 8.06 8.04 7.87 9/366 25852
/proc/meminfo: memFree=19756240/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=2867368 CPUtime=9756.51 cores=0-7
/proc/25796/stat : 25796 (plingeling) S 25794 25796 25304 0 -1 4202496 2725693 1253 0 0 975258 393 0 0 25 0 9 0 734772196 2936184832 675468 33554432000 4194304 4417452 140733542182848 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25796/statm: 716842 675468 180 55 0 714728 0
[pid=25796/tid=25804] ppid=25794 vsize=2867368 CPUtime=1204.09 cores=0-7
/proc/25796/task/25804/stat : 25804 (plingeling) R 25794 25796 25304 0 -1 4202560 259292 1253 0 0 120367 42 0 0 25 0 9 0 734774581 2936184832 675468 33554432000 4194304 4417452 140733542182848 18446744073709551615 4232787 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25796/tid=25805] ppid=25794 vsize=2867368 CPUtime=1218.4 cores=0-7
/proc/25796/task/25805/stat : 25805 (plingeling) R 25794 25796 25304 0 -1 4202560 252687 1253 0 0 121803 37 0 0 25 0 9 0 734774581 2936184832 675468 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=25796/tid=25806] ppid=25794 vsize=2867368 CPUtime=1218.44 cores=0-7
/proc/25796/task/25806/stat : 25806 (plingeling) R 25794 25796 25304 0 -1 4202560 257980 1253 0 0 121804 40 0 0 25 0 9 0 734774581 2936184832 675468 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25796/tid=25807] ppid=25794 vsize=2867368 CPUtime=1218.45 cores=0-7
/proc/25796/task/25807/stat : 25807 (plingeling) R 25794 25796 25304 0 -1 4202560 16001 1253 0 0 121842 3 0 0 25 0 9 0 734774581 2936184832 675468 33554432000 4194304 4417452 140733542182848 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=25796/tid=25808] ppid=25794 vsize=2867368 CPUtime=1218.21 cores=0-7
/proc/25796/task/25808/stat : 25808 (plingeling) R 25794 25796 25304 0 -1 4202560 312476 1253 0 0 121767 54 0 0 25 0 9 0 734774581 2936184832 675468 33554432000 4194304 4417452 140733542182848 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=25796/tid=25809] ppid=25794 vsize=2867368 CPUtime=1218.19 cores=0-7
/proc/25796/task/25809/stat : 25809 (plingeling) R 25794 25796 25304 0 -1 4202560 269149 1253 0 0 121777 42 0 0 25 0 9 0 734774582 2936184832 675468 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25796/tid=25810] ppid=25794 vsize=2867368 CPUtime=1218.42 cores=0-7
/proc/25796/task/25810/stat : 25810 (plingeling) R 25794 25796 25304 0 -1 4202560 246063 1253 0 0 121808 34 0 0 25 0 9 0 734774582 2936184832 675468 33554432000 4194304 4417452 140733542182848 18446744073709551615 4234671 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25796/tid=25811] ppid=25794 vsize=2867368 CPUtime=1218.39 cores=0-7
/proc/25796/task/25811/stat : 25811 (plingeling) R 25794 25796 25304 0 -1 4202560 304306 1253 0 0 121797 42 0 0 25 0 9 0 734774582 2936184832 675468 33554432000 4194304 4417452 140733542182848 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 9756.51
Current children cumulated vsize (KiB) 2867368

Solver just ended. Dumping a history of the last processes samples

[startup+1242.7 s]
/proc/loadavg: 8.06 8.04 7.87 9/366 25852
/proc/meminfo: memFree=19756240/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=2866472 CPUtime=9759.71 cores=0-7
/proc/25796/stat : 25796 (plingeling) S 25794 25796 25304 0 -1 4202496 2725693 1253 0 0 975578 393 0 0 25 0 9 0 734772196 2935267328 675244 33554432000 4194304 4417452 140733542182848 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25796/statm: 716618 675244 180 55 0 714504 0
[pid=25796/tid=25804] ppid=25794 vsize=2866472 CPUtime=1204.49 cores=0-7
/proc/25796/task/25804/stat : 25804 (plingeling) R 25794 25796 25304 0 -1 4202560 259292 1253 0 0 120407 42 0 0 25 0 9 0 734774581 2935267328 675244 33554432000 4194304 4417452 140733542182848 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25796/tid=25805] ppid=25794 vsize=2866472 CPUtime=1218.8 cores=0-7
/proc/25796/task/25805/stat : 25805 (plingeling) R 25794 25796 25304 0 -1 4202560 252687 1253 0 0 121843 37 0 0 25 0 9 0 734774581 2935267328 675244 33554432000 4194304 4417452 140733542182848 18446744073709551615 4232741 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=25796/tid=25806] ppid=25794 vsize=2866472 CPUtime=1218.84 cores=0-7
/proc/25796/task/25806/stat : 25806 (plingeling) R 25794 25796 25304 0 -1 4202560 257980 1253 0 0 121844 40 0 0 25 0 9 0 734774581 2935267328 675244 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25796/tid=25807] ppid=25794 vsize=2866472 CPUtime=1218.85 cores=0-7
/proc/25796/task/25807/stat : 25807 (plingeling) R 25794 25796 25304 0 -1 4202560 16001 1253 0 0 121882 3 0 0 25 0 9 0 734774581 2935267328 675244 33554432000 4194304 4417452 140733542182848 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=25796/tid=25808] ppid=25794 vsize=2866472 CPUtime=1218.61 cores=0-7
/proc/25796/task/25808/stat : 25808 (plingeling) R 25794 25796 25304 0 -1 4202560 312476 1253 0 0 121807 54 0 0 25 0 9 0 734774581 2935267328 675244 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=25796/tid=25809] ppid=25794 vsize=2866472 CPUtime=1218.59 cores=0-7
/proc/25796/task/25809/stat : 25809 (plingeling) R 25794 25796 25304 0 -1 4202560 269149 1253 0 0 121817 42 0 0 25 0 9 0 734774582 2935267328 675244 33554432000 4194304 4417452 140733542182848 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25796/tid=25810] ppid=25794 vsize=2866472 CPUtime=1218.82 cores=0-7
/proc/25796/task/25810/stat : 25810 (plingeling) R 25794 25796 25304 0 -1 4202560 246063 1253 0 0 121848 34 0 0 25 0 9 0 734774582 2935267328 675244 33554432000 4194304 4417452 140733542182848 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25796/tid=25811] ppid=25794 vsize=2866472 CPUtime=1218.79 cores=0-7
/proc/25796/task/25811/stat : 25811 (plingeling) R 25794 25796 25304 0 -1 4202560 304306 1253 0 0 121837 42 0 0 25 0 9 0 734774582 2935267328 675244 33554432000 4194304 4417452 140733542182848 18446744073709551615 4236971 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 9759.71
Current children cumulated vsize (KiB) 2866472

[startup+1243.1 s]
/proc/loadavg: 8.06 8.04 7.87 9/366 25852
/proc/meminfo: memFree=19756612/32950928 swapFree=65730864/67111528
[pid=25796] ppid=25794 vsize=365008 CPUtime=9762.15 cores=0-7
/proc/25796/stat : 25796 (plingeling) R 25794 25796 25304 0 -1 4202496 2725694 1253 0 0 975813 402 0 0 17 0 1 0 734772196 373768192 33943 33554432000 4194304 4417452 140733542182848 18446744073709551615 260905419882 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/25796/statm: 91252 33943 181 55 0 89138 0
Current children cumulated CPU time (s) 9762.15
Current children cumulated vsize (KiB) 365008

Child status: 20
Real time (s): 1243.11
CPU time (s): 9762.18
CPU user time (s): 9758.14
CPU system time (s): 4.03839
CPU usage (%): 785.304
Max. virtual memory (cumulated for all children) (KiB): 3950296

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 9758.14
system time used= 4.03839
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2726950
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 393
involuntary context switches= 47071

runsolver used 3.9364 second user time and 10.7404 second system time

The end

Launcher Data 

Begin job on node116 at 2011-05-15 09:43:35
IDJOB=3372971
IDBENCH=88158
IDSOLVER=1851
FILE ID=node116/3372971-1305445414
RUNJOBID= node116-1305442902-25318
PBS_JOBID= 13324834
Free space on /tmp= 72952 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT-Race-2010-CNF/hardware-verification/velev/1dlx_c_iq60_a.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372971-1305445414/watcher-3372971-1305445414 -o /tmp/evaluation-result-3372971-1305445414/solver-3372971-1305445414 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3372971-1305445414.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 1e60a63f1ba68b9b3c423ca413b4fa2b
RANDOM SEED=1847514696

node116.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      22457884 kB
Buffers:         93688 kB
Cached:         880544 kB
SwapCached:      12356 kB
Active:        9898988 kB
Inactive:       367532 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      22457884 kB
SwapTotal:    67111528 kB
SwapFree:     65730864 kB
Dirty:          225448 kB
Writeback:           0 kB
AnonPages:     9291036 kB
Mapped:          15420 kB
Slab:           115668 kB
PageTables:      44480 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 82744056 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72952 MiB
End job on node116 at 2011-05-15 10:04:18