Trace number 3371419

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.29 1203.56

General information on the benchmark

NameSAT02/industrial/aloul/
Homer/homer17.shuffled.cnf
MD5SUMca62f6128dc3cad7229c7b9ee33a7492
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark0.484925
Satisfiable
(Un)Satisfiability was proved
Number of variables286
Number of clauses1742
Sum of the clauses size3718
Maximum clause length11
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 21716
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 526

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371419-1305432558/watcher-3371419-1305432558 -o /tmp/evaluation-result-3371419-1305432558/solver-3371419-1305432558 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371419-1305432558.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.94 1.30 1.67 2/332 31060
/proc/meminfo: memFree=23203316/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=9004 CPUtime=0 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) R 31058 31060 22057 0 -1 4202496 339 1256 0 1 0 0 0 0 23 0 1 0 635671428 9220096 269 33554432000 4194304 4417452 140734050326512 18446744073709551615 270338383616 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/31060/statm: 2251 269 147 55 0 137 0
[pid=31060/tid=31069] ppid=31058 vsize=49976 CPUtime=0 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 16 1256 0 1 0 0 0 0 24 0 5 0 635671429 51175424 293 33554432000 4194304 4417452 140734050326512 18446744073709551615 4310822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=49976 CPUtime=0 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 0 1256 0 1 0 0 0 0 24 0 5 0 635671429 51175424 293 33554432000 4194304 4417452 140734050326512 18446744073709551615 270338440177 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=49976 CPUtime=0 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 0 1256 0 1 0 0 0 0 24 0 5 0 635671429 51175424 293 33554432000 4194304 4417452 140734050326512 18446744073709551615 270338440177 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=49976 CPUtime=0 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 0 1256 0 1 0 0 0 0 24 0 5 0 635671429 51175424 293 33554432000 4194304 4417452 140734050326512 18446744073709551615 270338440177 0 0 4096 17442 0 0 0 -1 3 0 0 0

[startup+0.058537 s]
/proc/loadavg: 0.94 1.30 1.67 2/332 31060
/proc/meminfo: memFree=23203316/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=52564 CPUtime=0.16 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 1052 1256 0 1 16 0 0 0 23 0 5 0 635671428 53825536 928 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 13141 928 180 55 0 11027 0
[pid=31060/tid=31069] ppid=31058 vsize=52564 CPUtime=0.03 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 170 1256 0 1 3 0 0 0 25 0 5 0 635671429 53825536 928 33554432000 4194304 4417452 140734050326512 18446744073709551615 18446744073699065918 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=52564 CPUtime=0.03 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 114 1256 0 1 3 0 0 0 25 0 5 0 635671429 53825536 928 33554432000 4194304 4417452 140734050326512 18446744073709551615 4310106 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=52564 CPUtime=0.04 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 151 1256 0 1 4 0 0 0 25 0 5 0 635671429 53825536 928 33554432000 4194304 4417452 140734050326512 18446744073709551615 18446744073699065939 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=52564 CPUtime=0.04 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 266 1256 0 1 4 0 0 0 25 0 5 0 635671429 53825536 928 33554432000 4194304 4417452 140734050326512 18446744073709551615 18446744073699065939 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 0.16
Current children cumulated vsize (KiB) 52564

[startup+0.100529 s]
/proc/loadavg: 0.94 1.30 1.67 2/332 31060
/proc/meminfo: memFree=23203316/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=118080 CPUtime=0.33 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 1241 1256 0 1 33 0 0 0 23 0 5 0 635671428 120913920 941 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 29520 941 180 55 0 27406 0
[pid=31060/tid=31069] ppid=31058 vsize=118080 CPUtime=0.07 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 170 1256 0 1 7 0 0 0 25 0 5 0 635671429 120913920 941 33554432000 4194304 4417452 140734050326512 18446744073709551615 4310184 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=118080 CPUtime=0.07 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 115 1256 0 1 7 0 0 0 25 0 5 0 635671429 120913920 941 33554432000 4194304 4417452 140734050326512 18446744073709551615 4312930 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=118080 CPUtime=0.08 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 176 1256 0 1 8 0 0 0 25 0 5 0 635671429 120913920 941 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=118080 CPUtime=0.08 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 429 1256 0 1 8 0 0 0 25 0 5 0 635671429 120913920 941 33554432000 4194304 4417452 140734050326512 18446744073709551615 4366141 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 0.33
Current children cumulated vsize (KiB) 118080

[startup+0.300491 s]
/proc/loadavg: 0.94 1.30 1.67 2/332 31060
/proc/meminfo: memFree=23203316/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=121848 CPUtime=1.12 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 3461 1256 0 1 112 0 0 0 23 0 5 0 635671428 124772352 1901 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 30462 1901 180 55 0 28348 0
[pid=31060/tid=31069] ppid=31058 vsize=121848 CPUtime=0.27 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 817 1256 0 1 27 0 0 0 25 0 5 0 635671429 124772352 1901 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364558 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=121848 CPUtime=0.27 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 543 1256 0 1 27 0 0 0 25 0 5 0 635671429 124772352 1901 33554432000 4194304 4417452 140734050326512 18446744073709551615 4356992 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=121848 CPUtime=0.28 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 766 1256 0 1 28 0 0 0 25 0 5 0 635671429 124772352 1901 33554432000 4194304 4417452 140734050326512 18446744073709551615 4366087 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=121848 CPUtime=0.28 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 984 1256 0 1 28 0 0 0 25 0 5 0 635671429 124772352 1901 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1.12
Current children cumulated vsize (KiB) 121848

[startup+0.700412 s]
/proc/loadavg: 0.94 1.30 1.67 2/332 31060
/proc/meminfo: memFree=23203316/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=126544 CPUtime=2.72 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 7212 1256 0 1 272 0 0 0 23 0 5 0 635671428 129581056 3075 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 31636 3075 180 55 0 29522 0
[pid=31060/tid=31069] ppid=31058 vsize=126544 CPUtime=0.67 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 1801 1256 0 1 67 0 0 0 25 0 5 0 635671429 129581056 3075 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=126544 CPUtime=0.67 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 1238 1256 0 1 67 0 0 0 25 0 5 0 635671429 129581056 3075 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=126544 CPUtime=0.68 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 1827 1256 0 1 68 0 0 0 25 0 5 0 635671429 129581056 3075 33554432000 4194304 4417452 140734050326512 18446744073709551615 4356995 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=126544 CPUtime=0.68 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 1995 1256 0 1 68 0 0 0 25 0 5 0 635671429 129581056 3075 33554432000 4194304 4417452 140734050326512 18446744073709551615 4236812 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2.72
Current children cumulated vsize (KiB) 126544

[startup+1.50026 s]
/proc/loadavg: 0.94 1.30 1.67 10/339 31075
/proc/meminfo: memFree=23177548/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=128592 CPUtime=5.91 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 15759 1256 0 1 589 2 0 0 23 0 5 0 635671428 131678208 3578 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 32148 3578 180 55 0 30034 0
[pid=31060/tid=31069] ppid=31058 vsize=128592 CPUtime=1.46 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 4382 1256 0 1 146 0 0 0 25 0 5 0 635671429 131678208 3578 33554432000 4194304 4417452 140734050326512 18446744073709551615 4251167 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=128592 CPUtime=1.47 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 3616 1256 0 1 147 0 0 0 25 0 5 0 635671429 131678208 3578 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=128592 CPUtime=1.48 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 4041 1256 0 1 148 0 0 0 25 0 5 0 635671429 131678208 3578 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=128592 CPUtime=1.46 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 3369 1256 0 1 146 0 0 0 25 0 5 0 635671429 131678208 3578 33554432000 4194304 4417452 140734050326512 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.91
Current children cumulated vsize (KiB) 128592

[startup+3.10094 s]
/proc/loadavg: 0.94 1.30 1.67 10/339 31075
/proc/meminfo: memFree=23168740/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=138244 CPUtime=12.3 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 28338 1256 0 1 1226 4 0 0 23 0 5 0 635671428 141561856 5968 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 34561 5968 180 55 0 32447 0
[pid=31060/tid=31069] ppid=31058 vsize=138244 CPUtime=3.06 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 7661 1256 0 1 306 0 0 0 25 0 5 0 635671429 141561856 5968 33554432000 4194304 4417452 140734050326512 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=138244 CPUtime=3.07 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 7547 1256 0 1 307 0 0 0 25 0 5 0 635671429 141561856 5968 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=138244 CPUtime=3.08 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 7167 1256 0 1 307 1 0 0 25 0 5 0 635671429 141561856 5968 33554432000 4194304 4417452 140734050326512 18446744073709551615 4232832 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=138244 CPUtime=3.06 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 5612 1256 0 1 305 1 0 0 25 0 5 0 635671429 141561856 5968 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.3
Current children cumulated vsize (KiB) 138244

[startup+6.3003 s]
/proc/loadavg: 1.50 1.41 1.71 10/339 31075
/proc/meminfo: memFree=23142676/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=148896 CPUtime=25.06 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 52540 1256 0 1 2498 8 0 0 23 0 5 0 635671428 152469504 8663 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 37224 8663 180 55 0 35110 0
[pid=31060/tid=31069] ppid=31058 vsize=148896 CPUtime=6.26 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 13712 1256 0 1 625 1 0 0 25 0 5 0 635671429 152469504 8663 33554432000 4194304 4417452 140734050326512 18446744073709551615 4366188 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=148896 CPUtime=6.27 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 15160 1256 0 1 625 2 0 0 25 0 5 0 635671429 152469504 8663 33554432000 4194304 4417452 140734050326512 18446744073709551615 4250697 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=148896 CPUtime=6.28 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 13453 1256 0 1 626 2 0 0 25 0 5 0 635671429 152469504 8663 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=148896 CPUtime=6.22 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 9864 1256 0 1 621 1 0 0 25 0 5 0 635671429 152469504 8663 33554432000 4194304 4417452 140734050326512 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 25.06
Current children cumulated vsize (KiB) 148896

[startup+12.701 s]
/proc/loadavg: 2.02 1.52 1.74 10/339 31075
/proc/meminfo: memFree=23121120/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=162248 CPUtime=50.6 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 91850 1256 0 1 5044 16 0 0 23 0 5 0 635671428 166141952 12001 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 40562 12001 180 55 0 38448 0
[pid=31060/tid=31069] ppid=31058 vsize=162248 CPUtime=12.66 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 24810 1256 0 1 1263 3 0 0 25 0 5 0 635671429 166141952 12001 33554432000 4194304 4417452 140734050326512 18446744073709551615 4354524 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=162248 CPUtime=12.67 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 26327 1256 0 1 1263 4 0 0 25 0 5 0 635671429 166141952 12001 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=162248 CPUtime=12.68 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 24345 1256 0 1 1264 4 0 0 25 0 5 0 635671429 166141952 12001 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=162248 CPUtime=12.55 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 16017 1256 0 1 1252 3 0 0 25 0 5 0 635671429 166141952 12001 33554432000 4194304 4417452 140734050326512 18446744073709551615 4250502 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.6
Current children cumulated vsize (KiB) 162248

[startup+25.5005 s]
/proc/loadavg: 3.35 1.84 1.84 10/339 31076
/proc/meminfo: memFree=23073500/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=177392 CPUtime=101.65 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 153399 1256 0 1 10140 25 0 0 23 0 5 0 635671428 181649408 15787 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 44348 15787 180 55 0 42234 0
[pid=31060/tid=31069] ppid=31058 vsize=177392 CPUtime=25.46 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 41364 1256 0 1 2540 6 0 0 25 0 5 0 635671429 181649408 15787 33554432000 4194304 4417452 140734050326512 18446744073709551615 4263109 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=177392 CPUtime=25.47 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 41176 1256 0 1 2541 6 0 0 25 0 5 0 635671429 181649408 15787 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=177392 CPUtime=25.47 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 44187 1256 0 1 2541 6 0 0 25 0 5 0 635671429 181649408 15787 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364558 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=177392 CPUtime=25.2 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 26321 1256 0 1 2516 4 0 0 25 0 5 0 635671429 181649408 15787 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364809 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.65
Current children cumulated vsize (KiB) 177392

[startup+51.1004 s]
/proc/loadavg: 5.01 2.35 2.01 10/339 31076
/proc/meminfo: memFree=23020896/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=227364 CPUtime=203.78 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 261958 1256 0 1 20336 42 0 0 23 0 5 0 635671428 232820736 28258 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 56841 28258 180 55 0 54727 0
[pid=31060/tid=31069] ppid=31058 vsize=227364 CPUtime=51.07 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 78791 1256 0 1 5095 12 0 0 25 0 5 0 635671429 232820736 28258 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=227364 CPUtime=51.07 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 66525 1256 0 1 5096 11 0 0 25 0 5 0 635671429 232820736 28258 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=227364 CPUtime=51.08 cores=1,3,5,7

################
# More data... #
################

[pid=31060/tid=31070] ppid=31058 vsize=744168 CPUtime=762.31 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 389370 1256 0 1 76166 65 0 0 25 0 5 0 635671429 762028032 157449 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=744168 CPUtime=762.32 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 453356 1256 0 1 76157 75 0 0 25 0 5 0 635671429 762028032 157449 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364850 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=744168 CPUtime=754.01 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 222377 1256 0 1 75361 40 0 0 25 0 5 0 635671429 762028032 157449 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364735 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3041
Current children cumulated vsize (KiB) 744168

[startup+822.3 s]
/proc/loadavg: 8.18 7.69 5.43 9/339 31167
/proc/meminfo: memFree=20812256/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=945684 CPUtime=3280.31 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 1495838 1256 0 1 327780 251 0 0 23 0 5 0 635671428 968380416 207828 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 236421 207828 180 55 0 234307 0
[pid=31060/tid=31069] ppid=31058 vsize=945684 CPUtime=822.3 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 395609 1256 0 1 82167 63 0 0 25 0 5 0 635671429 968380416 207828 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=945684 CPUtime=822.3 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 396646 1256 0 1 82164 66 0 0 25 0 5 0 635671429 968380416 207828 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=945684 CPUtime=822.31 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 463590 1256 0 1 82155 76 0 0 25 0 5 0 635671429 968380416 207828 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=945684 CPUtime=813.36 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 239642 1256 0 1 81293 43 0 0 25 0 5 0 635671429 968380416 207828 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3280.31
Current children cumulated vsize (KiB) 945684

[startup+882.3 s]
/proc/loadavg: 8.17 7.77 5.60 9/339 31168
/proc/meminfo: memFree=20935712/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=856412 CPUtime=3519.66 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 1564803 1256 0 1 351705 261 0 0 23 0 5 0 635671428 876965888 185542 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 214103 185542 180 55 0 211989 0
[pid=31060/tid=31069] ppid=31058 vsize=856412 CPUtime=882.31 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 412780 1256 0 1 88164 67 0 0 25 0 5 0 635671429 876965888 185542 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=856412 CPUtime=882.3 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 396646 1256 0 1 88164 66 0 0 25 0 5 0 635671429 876965888 185542 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364896 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=856412 CPUtime=882.32 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 507192 1256 0 1 88149 83 0 0 25 0 5 0 635671429 876965888 185542 33554432000 4194304 4417452 140734050326512 18446744073709551615 4365078 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=856412 CPUtime=872.7 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 247834 1256 0 1 87226 44 0 0 25 0 5 0 635671429 876965888 185542 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3519.66
Current children cumulated vsize (KiB) 856412

[startup+942.301 s]
/proc/loadavg: 8.23 7.86 5.76 9/339 31172
/proc/meminfo: memFree=20773600/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=966088 CPUtime=3759.02 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 1611678 1256 0 1 375633 269 0 0 23 0 5 0 635671428 989274112 212961 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 241522 212961 180 55 0 239408 0
[pid=31060/tid=31069] ppid=31058 vsize=966088 CPUtime=942.31 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 412780 1256 0 1 94164 67 0 0 25 0 5 0 635671429 989274112 212961 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=966088 CPUtime=942.31 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 443521 1256 0 1 94157 74 0 0 25 0 5 0 635671429 989274112 212961 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=966088 CPUtime=942.32 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 507192 1256 0 1 94149 83 0 0 25 0 5 0 635671429 989274112 212961 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=966088 CPUtime=932.05 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 247834 1256 0 1 93161 44 0 0 25 0 5 0 635671429 989274112 212961 33554432000 4194304 4417452 140734050326512 18446744073709551615 4366806 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3759.02
Current children cumulated vsize (KiB) 966088

[startup+1002.3 s]
/proc/loadavg: 8.46 8.00 5.94 9/339 31175
/proc/meminfo: memFree=20701536/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=1036832 CPUtime=3997.54 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 1704458 1256 0 1 399467 287 0 0 23 0 5 0 635671428 1061715968 230647 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 259208 230647 180 55 0 257094 0
[pid=31060/tid=31069] ppid=31058 vsize=1036832 CPUtime=1002.31 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 443078 1256 0 1 100158 73 0 0 25 0 5 0 635671429 1061715968 230647 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=1036832 CPUtime=1002.32 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 475345 1256 0 1 100151 81 0 0 25 0 5 0 635671429 1061715968 230647 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=1036832 CPUtime=1001.99 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 526426 1256 0 1 100114 85 0 0 25 0 5 0 635671429 1061715968 230647 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=1036832 CPUtime=990.89 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 259258 1256 0 1 99043 46 0 0 25 0 5 0 635671429 1061715968 230647 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3997.54
Current children cumulated vsize (KiB) 1036832

[startup+1062.3 s]
/proc/loadavg: 8.22 8.01 6.07 9/339 31176
/proc/meminfo: memFree=20757292/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=1021484 CPUtime=4236.91 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 1764201 1256 0 1 423393 298 0 0 23 0 5 0 635671428 1045999616 226778 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 255371 226778 180 55 0 253257 0
[pid=31060/tid=31069] ppid=31058 vsize=1021484 CPUtime=1062.31 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 445075 1256 0 1 106158 73 0 0 25 0 5 0 635671429 1045999616 226778 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=1021484 CPUtime=1062.32 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 475345 1256 0 1 106151 81 0 0 25 0 5 0 635671429 1045999616 226778 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=1021484 CPUtime=1061.99 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 544156 1256 0 1 106111 88 0 0 25 0 5 0 635671429 1045999616 226778 33554432000 4194304 4417452 140734050326512 18446744073709551615 4366400 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=1021484 CPUtime=1050.25 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 299274 1256 0 1 104972 53 0 0 25 0 5 0 635671429 1045999616 226778 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4236.91
Current children cumulated vsize (KiB) 1021484

[startup+1122.3 s]
/proc/loadavg: 8.23 8.05 6.21 9/339 31178
/proc/meminfo: memFree=20835876/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=1047252 CPUtime=4476.26 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 1794453 1256 0 1 447323 303 0 0 23 0 5 0 635671428 1072386048 232966 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 261813 232966 180 55 0 259699 0
[pid=31060/tid=31069] ppid=31058 vsize=1047252 CPUtime=1122.31 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 445075 1256 0 1 112158 73 0 0 25 0 5 0 635671429 1072386048 232966 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=1047252 CPUtime=1122.32 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 475345 1256 0 1 112151 81 0 0 25 0 5 0 635671429 1072386048 232966 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=1047252 CPUtime=1121.99 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 566216 1256 0 1 112107 92 0 0 25 0 5 0 635671429 1072386048 232966 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=1047252 CPUtime=1109.6 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 307466 1256 0 1 110905 55 0 0 25 0 5 0 635671429 1072386048 232966 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4476.26
Current children cumulated vsize (KiB) 1047252

[startup+1182.3 s]
/proc/loadavg: 8.13 8.06 6.32 9/339 31179
/proc/meminfo: memFree=20691752/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=1096000 CPUtime=4715.62 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 1814832 1256 0 1 471255 307 0 0 23 0 5 0 635671428 1122304000 245153 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 274000 245153 180 55 0 271886 0
[pid=31060/tid=31069] ppid=31058 vsize=1096000 CPUtime=1182.31 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 457262 1256 0 1 118155 76 0 0 25 0 5 0 635671429 1122304000 245153 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=1096000 CPUtime=1182.32 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 475345 1256 0 1 118151 81 0 0 25 0 5 0 635671429 1122304000 245153 33554432000 4194304 4417452 140734050326512 18446744073709551615 4344164 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=1096000 CPUtime=1181.99 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 566216 1256 0 1 118107 92 0 0 25 0 5 0 635671429 1122304000 245153 33554432000 4194304 4417452 140734050326512 18446744073709551615 4365092 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=1096000 CPUtime=1168.95 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 315658 1256 0 1 116839 56 0 0 25 0 5 0 635671429 1122304000 245153 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4715.62
Current children cumulated vsize (KiB) 1096000



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.5 s]
/proc/loadavg: 8.08 8.05 6.36 10/339 31180
/proc/meminfo: memFree=20674244/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=1096000 CPUtime=4800.22 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 1823024 1256 0 1 479714 308 0 0 23 0 5 0 635671428 1122304000 245153 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 274000 245153 180 55 0 271886 0
[pid=31060/tid=31069] ppid=31058 vsize=1096000 CPUtime=1203.53 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 457262 1256 0 1 120277 76 0 0 25 0 5 0 635671429 1122304000 245153 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=1096000 CPUtime=1203.54 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 475345 1256 0 1 120273 81 0 0 25 0 5 0 635671429 1122304000 245153 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=1096000 CPUtime=1203.2 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 566216 1256 0 1 120228 92 0 0 25 0 5 0 635671429 1122304000 245153 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=1096000 CPUtime=1189.91 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 323850 1256 0 1 118935 56 0 0 25 0 5 0 635671429 1122304000 245153 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.22
Current children cumulated vsize (KiB) 1096000

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.5 s]
/proc/loadavg: 8.08 8.05 6.36 10/339 31180
/proc/meminfo: memFree=20674244/32951124 swapFree=59367104/67111528
[pid=31060] ppid=31058 vsize=1096000 CPUtime=4800.22 cores=1,3,5,7
/proc/31060/stat : 31060 (plingeling) S 31058 31060 22057 0 -1 4202496 1823024 1256 0 1 479714 308 0 0 23 0 5 0 635671428 1122304000 245153 33554432000 4194304 4417452 140734050326512 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/31060/statm: 274000 245153 180 55 0 271886 0
[pid=31060/tid=31069] ppid=31058 vsize=1096000 CPUtime=1203.53 cores=1,3,5,7
/proc/31060/task/31069/stat : 31069 (plingeling) R 31058 31060 22057 0 -1 4202560 457262 1256 0 1 120277 76 0 0 25 0 5 0 635671429 1122304000 245153 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31060/tid=31070] ppid=31058 vsize=1096000 CPUtime=1203.54 cores=1,3,5,7
/proc/31060/task/31070/stat : 31070 (plingeling) R 31058 31060 22057 0 -1 4202560 475345 1256 0 1 120273 81 0 0 25 0 5 0 635671429 1122304000 245153 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31060/tid=31071] ppid=31058 vsize=1096000 CPUtime=1203.2 cores=1,3,5,7
/proc/31060/task/31071/stat : 31071 (plingeling) R 31058 31060 22057 0 -1 4202560 566216 1256 0 1 120228 92 0 0 25 0 5 0 635671429 1122304000 245153 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31060/tid=31072] ppid=31058 vsize=1096000 CPUtime=1189.91 cores=1,3,5,7
/proc/31060/task/31072/stat : 31072 (plingeling) R 31058 31060 22057 0 -1 4202560 323850 1256 0 1 118935 56 0 0 25 0 5 0 635671429 1122304000 245153 33554432000 4194304 4417452 140734050326512 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.22
Current children cumulated vsize (KiB) 1096000

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.56
CPU time (s): 4800.29
CPU user time (s): 4797.17
CPU system time (s): 3.12052
CPU usage (%): 398.842
Max. virtual memory (cumulated for all children) (KiB): 1144828

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.17
system time used= 3.12052
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1824281
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 74
involuntary context switches= 26449

runsolver used 3.50147 second user time and 9.64553 second system time

The end

Launcher Data 

Begin job on node143 at 2011-05-15 06:09:18
IDJOB=3371419
IDBENCH=87946
IDSOLVER=1848
FILE ID=node143/3371419-1305432558
RUNJOBID= node143-1305432558-31037
PBS_JOBID= 13324594
Free space on /tmp= 73552 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT02/industrial/aloul/Homer/homer17.shuffled.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371419-1305432558/watcher-3371419-1305432558 -o /tmp/evaluation-result-3371419-1305432558/solver-3371419-1305432558 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371419-1305432558.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= ca62f6128dc3cad7229c7b9ee33a7492
RANDOM SEED=1238252377

node143.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23203596 kB
Buffers:        185920 kB
Cached:         804012 kB
SwapCached:      67400 kB
Active:         148396 kB
Inactive:      9362812 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23203596 kB
SwapTotal:    67111528 kB
SwapFree:     59367104 kB
Dirty:             428 kB
Writeback:          20 kB
AnonPages:     8454024 kB
Mapped:          22384 kB
Slab:           118512 kB
PageTables:      52916 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 69995200 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73364 MiB
End job on node143 at 2011-05-15 06:29:24