Trace number 3385946

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 3903.22 1300.06

General information on the benchmark

NameSAT05/crafted/jarvisalo05/mod2c-rand3bip-sat/
mod2c-rand3bip-sat-250-3.shuffled-as.sat05-2535.cnf
MD5SUM22bc1843113c53c5f8a8f8cec42dcbe5
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark240.682
Satisfiable
(Un)Satisfiability was proved
Number of variables352
Number of clauses2492
Sum of the clauses size11032
Maximum clause length6
Minimum clause length4
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 41592
Number of clauses of size 5736
Number of clauses of size over 5164

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385946-1305583004/watcher-3385946-1305583004 -o /tmp/evaluation-result-3385946-1305583004/solver-3385946-1305583004 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3385946-1305583004.cnf -threads=4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.99 7.31 9.04 7/446 12248
/proc/meminfo: memFree=19278004/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=49880 CPUtime=0.01 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 604 0 0 0 1 0 0 0 25 0 4 0 650716289 51077120 541 33554432000 4194304 4625164 140736168550848 18446744073709551615 47805543442354 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/12248/statm: 12470 541 301 106 0 7973 0
[pid=12248/tid=12249] ppid=12246 vsize=49880 CPUtime=0 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 4 0 650716290 51077120 541 33554432000 4194304 4625164 140736168550848 18446744073709551615 47805543442354 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=49880 CPUtime=0 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 4 0 650716290 51077120 541 33554432000 4194304 4625164 140736168550848 18446744073709551615 233510353905 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=49880 CPUtime=0.01 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1 0 0 0 1 0 0 0 25 0 4 0 650716290 51077120 541 33554432000 4194304 4625164 140736168550848 18446744073709551615 47805543442354 0 0 4096 2 0 0 0 -1 3 0 0 0

[startup+0.075108 s]
/proc/loadavg: 5.99 7.31 9.04 7/446 12248
/proc/meminfo: memFree=19278004/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=182024 CPUtime=0.2 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) D 12246 12248 11674 0 -1 4202496 1165 0 0 0 20 0 0 0 25 0 4 0 650716289 186392576 1002 33554432000 4194304 4625164 140736168550848 18446744073709551615 233510340791 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 45506 1003 338 106 0 41009 0
[pid=12248/tid=12249] ppid=12246 vsize=182024 CPUtime=0.05 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 105 0 0 0 5 0 0 0 25 0 4 0 650716290 186392576 1003 33554432000 4194304 4625164 140736168550848 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=182024 CPUtime=0 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) D 12246 12248 11674 0 -1 4202560 21 0 0 0 0 0 0 0 25 0 4 0 650716290 186392576 1003 33554432000 4194304 4625164 140736168550848 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=182024 CPUtime=0.06 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) D 12246 12248 11674 0 -1 4202560 110 0 0 0 6 0 0 0 25 0 4 0 650716290 186392576 1003 33554432000 4194304 4625164 140736168550848 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 0.2
Current children cumulated vsize (KiB) 182024

[startup+0.809693 s]
/proc/loadavg: 5.99 7.31 9.04 7/446 12248
/proc/meminfo: memFree=19278004/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=251596 CPUtime=2.69 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 4632 0 0 0 269 0 0 0 25 0 4 0 650716289 257634304 2805 33554432000 4194304 4625164 140736168550848 18446744073709551615 233510340791 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12248/statm: 62899 2805 343 106 0 58402 0
[pid=12248/tid=12249] ppid=12246 vsize=251596 CPUtime=0.79 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) D 12246 12248 11674 0 -1 4202560 1149 0 0 0 79 0 0 0 18 0 4 0 650716290 257634304 2805 33554432000 4194304 4625164 140736168550848 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=251596 CPUtime=0.73 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 948 0 0 0 73 0 0 0 25 0 4 0 650716290 257634304 2805 33554432000 4194304 4625164 140736168550848 18446744073709551615 4352145 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=251596 CPUtime=0.8 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1301 0 0 0 80 0 0 0 18 0 4 0 650716290 257634304 2805 33554432000 4194304 4625164 140736168550848 18446744073709551615 4551127 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2.69
Current children cumulated vsize (KiB) 251596

[startup+0.900396 s]
/proc/loadavg: 5.99 7.31 9.04 7/446 12248
/proc/meminfo: memFree=19278004/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=251928 CPUtime=2.95 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 5066 0 0 0 295 0 0 0 25 0 4 0 650716289 257974272 3239 33554432000 4194304 4625164 140736168550848 18446744073709551615 4511874 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 62982 3239 343 106 0 58485 0
[pid=12248/tid=12249] ppid=12246 vsize=251928 CPUtime=0.84 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1199 0 0 0 84 0 0 0 19 0 4 0 650716290 257974272 3239 33554432000 4194304 4625164 140736168550848 18446744073709551615 4352592 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=251928 CPUtime=0.82 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1003 0 0 0 82 0 0 0 25 0 4 0 650716290 257974272 3239 33554432000 4194304 4625164 140736168550848 18446744073709551615 4467253 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=251928 CPUtime=0.89 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1393 0 0 0 89 0 0 0 18 0 4 0 650716290 257974272 3239 33554432000 4194304 4625164 140736168550848 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2.95
Current children cumulated vsize (KiB) 251928

[startup+1.00038 s]
/proc/loadavg: 5.99 7.31 9.04 7/446 12248
/proc/meminfo: memFree=19278004/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=253108 CPUtime=3.25 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 5723 0 0 0 325 0 0 0 25 0 4 0 650716289 259182592 3402 33554432000 4194304 4625164 140736168550848 18446744073709551615 4511941 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 63277 3402 343 106 0 58780 0
[pid=12248/tid=12249] ppid=12246 vsize=253108 CPUtime=0.92 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1241 0 0 0 92 0 0 0 19 0 4 0 650716290 259182592 3402 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351795 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=253108 CPUtime=0.92 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1306 0 0 0 92 0 0 0 25 0 4 0 650716290 259182592 3402 33554432000 4194304 4625164 140736168550848 18446744073709551615 4352208 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=253108 CPUtime=0.99 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1703 0 0 0 99 0 0 0 19 0 4 0 650716290 259182592 3402 33554432000 4194304 4625164 140736168550848 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3.25
Current children cumulated vsize (KiB) 253108

[startup+1.50029 s]
/proc/loadavg: 5.99 7.31 9.04 7/446 12248
/proc/meminfo: memFree=19278004/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=253856 CPUtime=4.76 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 6664 0 0 0 475 1 0 0 25 0 4 0 650716289 259948544 4265 33554432000 4194304 4625164 140736168550848 18446744073709551615 4512057 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 63464 4265 344 106 0 58967 0
[pid=12248/tid=12249] ppid=12246 vsize=253856 CPUtime=1.14 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1431 0 0 0 114 0 0 0 19 0 4 0 650716290 259948544 4265 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=253856 CPUtime=1.42 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1613 0 0 0 142 0 0 0 25 0 4 0 650716290 259948544 4265 33554432000 4194304 4625164 140736168550848 18446744073709551615 4352280 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=253856 CPUtime=1.49 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 2071 0 0 0 149 0 0 0 20 0 4 0 650716290 259948544 4265 33554432000 4194304 4625164 140736168550848 18446744073709551615 4547441 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4.76
Current children cumulated vsize (KiB) 253856

[startup+3.10101 s]
/proc/loadavg: 5.99 7.31 9.04 11/450 12252
/proc/meminfo: memFree=19252368/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=261224 CPUtime=9.53 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 11553 0 0 0 951 2 0 0 25 0 4 0 650716289 267493376 6857 33554432000 4194304 4625164 140736168550848 18446744073709551615 4356423 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 65306 6857 344 106 0 60809 0
[pid=12248/tid=12249] ppid=12246 vsize=261224 CPUtime=1.94 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 2402 0 0 0 194 0 0 0 21 0 4 0 650716290 267493376 6857 33554432000 4194304 4625164 140736168550848 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=261224 CPUtime=3.01 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 3388 0 0 0 301 0 0 0 25 0 4 0 650716290 267493376 6857 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=261224 CPUtime=3.09 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 3963 0 0 0 309 0 0 0 25 0 4 0 650716290 267493376 6857 33554432000 4194304 4625164 140736168550848 18446744073709551615 4551632 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 9.53
Current children cumulated vsize (KiB) 261224

[startup+6.30043 s]
/proc/loadavg: 6.31 7.35 9.04 11/450 12252
/proc/meminfo: memFree=19235504/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=270812 CPUtime=19.07 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 18196 0 0 0 1903 4 0 0 25 0 4 0 650716289 277311488 10363 33554432000 4194304 4625164 140736168550848 18446744073709551615 4509360 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 67703 10363 344 106 0 63206 0
[pid=12248/tid=12249] ppid=12246 vsize=270812 CPUtime=3.5 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 4538 0 0 0 349 1 0 0 25 0 4 0 650716290 277311488 10363 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=270812 CPUtime=5.86 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 5160 0 0 0 585 1 0 0 25 0 4 0 650716290 277311488 10363 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=270812 CPUtime=6.28 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 5933 0 0 0 627 1 0 0 25 0 4 0 650716290 277311488 10363 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351909 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 19.07
Current children cumulated vsize (KiB) 270812

[startup+12.7003 s]
/proc/loadavg: 6.61 7.39 9.05 11/450 12252
/proc/meminfo: memFree=19213828/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=283228 CPUtime=38.19 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 58255 0 0 0 3807 12 0 0 25 0 4 0 650716289 290025472 15045 33554432000 4194304 4625164 140736168550848 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 70807 15045 356 106 0 66310 0
[pid=12248/tid=12249] ppid=12246 vsize=283228 CPUtime=6.64 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 6763 0 0 0 662 2 0 0 25 0 4 0 650716290 290025472 15267 33554432000 4194304 4625164 140736168550848 18446744073709551615 4352728 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=283228 CPUtime=12.28 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 8184 0 0 0 1226 2 0 0 25 0 4 0 650716290 290025472 15267 33554432000 4194304 4625164 140736168550848 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=283228 CPUtime=11.79 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 39894 0 0 0 1172 7 0 0 25 0 4 0 650716290 290025472 15267 33554432000 4194304 4625164 140736168550848 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38.19
Current children cumulated vsize (KiB) 283228

[startup+25.501 s]
/proc/loadavg: 7.36 7.52 9.06 11/450 12252
/proc/meminfo: memFree=19176120/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=305368 CPUtime=76.38 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 106525 0 0 0 7616 22 0 0 25 0 4 0 650716289 312696832 23441 33554432000 4194304 4625164 140736168550848 18446744073709551615 4356581 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 76342 23441 402 106 0 71845 0
[pid=12248/tid=12249] ppid=12246 vsize=305368 CPUtime=12.93 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 9789 0 0 0 1291 2 0 0 25 0 4 0 650716290 312696832 23441 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351987 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=305368 CPUtime=25.06 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 12906 0 0 0 2503 3 0 0 25 0 4 0 650716290 312696832 23441 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=305368 CPUtime=18.28 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 77520 0 0 0 1814 14 0 0 25 0 4 0 650716290 312696832 23441 33554432000 4194304 4625164 140736168550848 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 76.38
Current children cumulated vsize (KiB) 305368

[startup+51.1004 s]
/proc/loadavg: 8.32 7.74 9.09 11/450 12253
/proc/meminfo: memFree=19129932/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=334344 CPUtime=152.8 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 142132 0 0 0 15249 31 0 0 25 0 4 0 650716289 342368256 34454 33554432000 4194304 4625164 140736168550848 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 83586 34454 355 106 0 79089 0
[pid=12248/tid=12249] ppid=12246 vsize=334344 CPUtime=25.81 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 14947 0 0 0 2578 3 0 0 25 0 4 0 650716290 342368256 34454 33554432000 4194304 4625164 140736168550848 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=334344 CPUtime=50.66 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 19498 0 0 0 5059 7 0 0 25 0 4 0 650716290 342368256 34454 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=334344 CPUtime=31 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 99286 0 0 0 3083 17 0 0 25 0 4 0 650716290 342368256 34454 33554432000 4194304 4625164 140736168550848 18446744073709551615 4278521 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 152.8
Current children cumulated vsize (KiB) 334344

[startup+102.3 s]
/proc/loadavg: 9.33 8.10 9.14 11/450 12254
/proc/meminfo: memFree=19039072/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=377592 CPUtime=305.6 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 174419 0 0 0 30516 44 0 0 25 0 4 0 650716289 386654208 51635 33554432000 4194304 4625164 140736168550848 18446744073709551615 4519341 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 94398 51635 367 106 0 89901 0
[pid=12248/tid=12249] ppid=12246 vsize=377592 CPUtime=51.42 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 16184 0 0 0 5138 4 0 0 25 0 4 0 650716290 386654208 51635 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=377592 CPUtime=101.87 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 39389 0 0 0 10170 17 0 0 25 0 4 0 650716290 386654208 51635 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=377592 CPUtime=56.6 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 105491 0 0 0 5640 20 0 0 25 0 4 0 650716290 386654208 51635 33554432000 4194304 4625164 140736168550848 18446744073709551615 4579376 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 305.6
Current children cumulated vsize (KiB) 377592

[startup+162.3 s]

################
# More data... #
################

[pid=12248] ppid=12246 vsize=856360 CPUtime=2275.87 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 1299076 0 0 0 227333 254 0 0 25 0 4 0 650716289 876912640 147089 33554432000 4194304 4625164 140736168550848 18446744073709551615 4519341 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 214090 147089 344 106 0 209593 0
[pid=12248/tid=12249] ppid=12246 vsize=856360 CPUtime=534.66 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 76409 0 0 0 53441 25 0 0 25 0 4 0 650716290 876912640 147089 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=856360 CPUtime=456.6 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 84730 0 0 0 45630 30 0 0 25 0 4 0 650716290 876912640 147089 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=856360 CPUtime=539.02 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1039352 0 0 0 53725 177 0 0 25 0 4 0 650716290 876912640 147089 33554432000 4194304 4625164 140736168550848 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2275.87
Current children cumulated vsize (KiB) 856360

[startup+822.3 s]
/proc/loadavg: 10.29 9.96 9.66 11/450 12283
/proc/meminfo: memFree=17329540/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=949392 CPUtime=2455.02 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 1325336 0 0 0 245241 261 0 0 25 0 4 0 650716289 972177408 154578 33554432000 4194304 4625164 140736168550848 18446744073709551615 4519319 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 237348 154578 344 106 0 232851 0
[pid=12248/tid=12249] ppid=12246 vsize=949392 CPUtime=564.66 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 77157 0 0 0 56439 27 0 0 25 0 4 0 650716290 972177408 154578 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=949392 CPUtime=486.62 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 86370 0 0 0 48630 32 0 0 25 0 4 0 650716290 972177408 154578 33554432000 4194304 4625164 140736168550848 18446744073709551615 4352739 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=949392 CPUtime=599.04 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1041595 0 0 0 59726 178 0 0 25 0 4 0 650716290 972177408 154578 33554432000 4194304 4625164 140736168550848 18446744073709551615 4547232 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2455.02
Current children cumulated vsize (KiB) 949392

[startup+882.3 s]
/proc/loadavg: 10.28 10.01 9.69 11/450 12284
/proc/meminfo: memFree=17174840/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=955304 CPUtime=2634.13 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 1339481 0 0 0 263147 266 0 0 25 0 4 0 650716289 978231296 158666 33554432000 4194304 4625164 140736168550848 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 238826 158666 344 106 0 234329 0
[pid=12248/tid=12249] ppid=12246 vsize=955304 CPUtime=594.65 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 77863 0 0 0 59437 28 0 0 25 0 4 0 650716290 978231296 158666 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=955304 CPUtime=516.61 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 87050 0 0 0 51628 33 0 0 25 0 4 0 650716290 978231296 158666 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=955304 CPUtime=659.02 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1043597 0 0 0 65724 178 0 0 25 0 4 0 650716290 978231296 158666 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351860 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2634.13
Current children cumulated vsize (KiB) 955304

[startup+942.317 s]
/proc/loadavg: 10.21 10.04 9.72 11/450 12286
/proc/meminfo: memFree=16942128/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=962336 CPUtime=2813.29 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 1354972 0 0 0 281058 271 0 0 25 0 4 0 650716289 985432064 164550 33554432000 4194304 4625164 140736168550848 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 240584 164550 344 106 0 236087 0
[pid=12248/tid=12249] ppid=12246 vsize=962336 CPUtime=624.65 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 78583 0 0 0 62436 29 0 0 25 0 4 0 650716290 985432064 164550 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=962336 CPUtime=546.63 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 87806 0 0 0 54629 34 0 0 25 0 4 0 650716290 985432064 164550 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=962336 CPUtime=719.04 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1046104 0 0 0 71725 179 0 0 25 0 4 0 650716290 985432064 164550 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2813.29
Current children cumulated vsize (KiB) 962336

[startup+1002.3 s]
/proc/loadavg: 10.18 10.06 9.75 11/450 12287
/proc/meminfo: memFree=16801692/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=977664 CPUtime=2992.37 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 1369653 0 0 0 298960 277 0 0 25 0 4 0 650716289 1001127936 172292 33554432000 4194304 4625164 140736168550848 18446744073709551615 4519334 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 244416 172292 344 106 0 239919 0
[pid=12248/tid=12249] ppid=12246 vsize=977664 CPUtime=654.66 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 79333 0 0 0 65436 30 0 0 25 0 4 0 650716290 1001127936 172292 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351981 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=977664 CPUtime=585.43 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 89628 0 0 0 58506 37 0 0 25 0 4 0 650716290 1001127936 172292 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=977664 CPUtime=770.23 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1050123 0 0 0 76844 179 0 0 25 0 4 0 650716290 1001127936 172292 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2992.37
Current children cumulated vsize (KiB) 977664

[startup+1062.3 s]
/proc/loadavg: 10.22 10.09 9.77 11/450 12289
/proc/meminfo: memFree=16652060/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=980688 CPUtime=3171.51 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 1375663 0 0 0 316870 281 0 0 25 0 4 0 650716289 1004224512 178191 33554432000 4194304 4625164 140736168550848 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 245172 178191 344 106 0 240675 0
[pid=12248/tid=12249] ppid=12246 vsize=980688 CPUtime=684.67 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 80492 0 0 0 68435 32 0 0 25 0 4 0 650716290 1004224512 178191 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=980688 CPUtime=615.42 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 90254 0 0 0 61505 37 0 0 25 0 4 0 650716290 1004224512 178191 33554432000 4194304 4625164 140736168550848 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=980688 CPUtime=830.23 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1051007 0 0 0 82844 179 0 0 25 0 4 0 650716290 1004224512 178191 33554432000 4194304 4625164 140736168550848 18446744073709551615 4352760 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3171.51
Current children cumulated vsize (KiB) 980688

[startup+1122.3 s]
/proc/loadavg: 10.08 10.07 9.78 11/450 12291
/proc/meminfo: memFree=16319808/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=1041432 CPUtime=3350.64 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 1408680 0 0 0 334777 287 0 0 25 0 4 0 650716289 1066426368 181624 33554432000 4194304 4625164 140736168550848 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 260358 181624 344 106 0 255861 0
[pid=12248/tid=12249] ppid=12246 vsize=1041432 CPUtime=714.68 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 81231 0 0 0 71434 34 0 0 25 0 4 0 650716290 1066426368 181624 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=1041432 CPUtime=645.42 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 90882 0 0 0 64504 38 0 0 25 0 4 0 650716290 1066426368 181624 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=1041432 CPUtime=890.24 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1066573 0 0 0 88842 182 0 0 25 0 4 0 650716290 1066426368 181624 33554432000 4194304 4625164 140736168550848 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3350.64
Current children cumulated vsize (KiB) 1041432

[startup+1182.3 s]
/proc/loadavg: 10.03 10.05 9.80 11/450 12292
/proc/meminfo: memFree=15941512/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=1047488 CPUtime=3529.76 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 1413739 0 0 0 352686 290 0 0 25 0 4 0 650716289 1072627712 186562 33554432000 4194304 4625164 140736168550848 18446744073709551615 4512312 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 261872 186562 344 106 0 257375 0
[pid=12248/tid=12249] ppid=12246 vsize=1047488 CPUtime=744.69 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 81926 0 0 0 74434 35 0 0 25 0 4 0 650716290 1072627712 186562 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=1047488 CPUtime=675.41 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 91418 0 0 0 67502 39 0 0 25 0 4 0 650716290 1072627712 186562 33554432000 4194304 4625164 140736168550848 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=1047488 CPUtime=950.24 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1068788 0 0 0 94842 182 0 0 25 0 4 0 650716290 1072627712 186562 33554432000 4194304 4625164 140736168550848 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3529.76
Current children cumulated vsize (KiB) 1047488

[startup+1242.3 s]
/proc/loadavg: 10.05 10.06 9.81 11/450 12294
/proc/meminfo: memFree=15729036/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=1123964 CPUtime=3708.9 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 1442648 0 0 0 370595 295 0 0 25 0 4 0 650716289 1150939136 200187 33554432000 4194304 4625164 140736168550848 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 280991 200187 344 106 0 276494 0
[pid=12248/tid=12249] ppid=12246 vsize=1123964 CPUtime=774.69 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 87799 0 0 0 77433 36 0 0 25 0 4 0 650716290 1150939136 200187 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=1123964 CPUtime=705.41 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 93462 0 0 0 70501 40 0 0 25 0 4 0 650716290 1150939136 200187 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=1123964 CPUtime=1010.25 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1070750 0 0 0 100842 183 0 0 25 0 4 0 650716290 1150939136 200187 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3708.9
Current children cumulated vsize (KiB) 1123964



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 8.69 9.74 9.72 7/441 12313
/proc/meminfo: memFree=19417144/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=1129920 CPUtime=3903.14 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 1447411 0 0 0 390015 299 0 0 25 0 4 0 650716289 1157038080 204931 33554432000 4194304 4625164 140736168550848 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 282480 204931 344 106 0 277983 0
[pid=12248/tid=12249] ppid=12246 vsize=1129920 CPUtime=814.51 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 88188 0 0 0 81415 36 0 0 25 0 4 0 650716290 1157038080 204932 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351814 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=1129920 CPUtime=745.27 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 94279 0 0 0 74485 42 0 0 25 0 4 0 650716290 1157038080 204932 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351818 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=1129920 CPUtime=1067.96 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1072985 0 0 0 106613 183 0 0 25 0 4 0 650716290 1157038080 204932 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3903.14
Current children cumulated vsize (KiB) 1129920

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1300 s]
/proc/loadavg: 8.69 9.74 9.72 7/441 12313
/proc/meminfo: memFree=19417144/32951124 swapFree=67023148/67111528
[pid=12248] ppid=12246 vsize=1129920 CPUtime=3903.14 cores=1,3,5,7
/proc/12248/stat : 12248 (SArTagnan) R 12246 12248 11674 0 -1 4202496 1447411 0 0 0 390015 299 0 0 25 0 4 0 650716289 1157038080 204931 33554432000 4194304 4625164 140736168550848 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12248/statm: 282480 204931 344 106 0 277983 0
[pid=12248/tid=12249] ppid=12246 vsize=1129920 CPUtime=814.51 cores=1,3,5,7
/proc/12248/task/12249/stat : 12249 (SArTagnan) R 12246 12248 11674 0 -1 4202560 88188 0 0 0 81415 36 0 0 25 0 4 0 650716290 1157038080 204932 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351814 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12248/tid=12250] ppid=12246 vsize=1129920 CPUtime=745.27 cores=1,3,5,7
/proc/12248/task/12250/stat : 12250 (SArTagnan) R 12246 12248 11674 0 -1 4202560 94279 0 0 0 74485 42 0 0 25 0 4 0 650716290 1157038080 204932 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351818 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12248/tid=12251] ppid=12246 vsize=1129920 CPUtime=1067.96 cores=1,3,5,7
/proc/12248/task/12251/stat : 12251 (SArTagnan) R 12246 12248 11674 0 -1 4202560 1072985 0 0 0 106613 183 0 0 25 0 4 0 650716290 1157038080 204932 33554432000 4194304 4625164 140736168550848 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3903.14
Current children cumulated vsize (KiB) 1129920

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1300.06
CPU time (s): 3903.22
CPU user time (s): 3900.2
CPU system time (s): 3.02554
CPU usage (%): 300.235
Max. virtual memory (cumulated for all children) (KiB): 1129920

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3900.2
system time used= 3.02554
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1447412
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 209
involuntary context switches= 36265

runsolver used 4.74028 second user time and 14.4548 second system time

The end

Launcher Data 

Begin job on node145 at 2011-05-16 23:56:44
IDJOB=3385946
IDBENCH=85355
IDSOLVER=1871
FILE ID=node145/3385946-1305583004
RUNJOBID= node145-1305582975-12165
PBS_JOBID= 13350223
Free space on /tmp= 73424 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT05/crafted/jarvisalo05/mod2c-rand3bip-sat/mod2c-rand3bip-sat-250-3.shuffled-as.sat05-2535.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385946-1305583004/watcher-3385946-1305583004 -o /tmp/evaluation-result-3385946-1305583004/solver-3385946-1305583004 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3385946-1305583004.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 22bc1843113c53c5f8a8f8cec42dcbe5
RANDOM SEED=172328491

node145.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      19278284 kB
Buffers:        112148 kB
Cached:        1192580 kB
SwapCached:       4796 kB
Active:        9800020 kB
Inactive:      3611264 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      19278284 kB
SwapTotal:    67111528 kB
SwapFree:     67023148 kB
Dirty:          214640 kB
Writeback:           0 kB
AnonPages:    12101900 kB
Mapped:          33952 kB
Slab:           138656 kB
PageTables:      56692 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 121464972 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73632 MiB
End job on node145 at 2011-05-17 00:18:26