Trace number 3386137

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 4800.39 1205.37

General information on the benchmark

Namecrafted/spence/sgen/
sgen3-n190-s29985700-unsat.cnf
MD5SUMf2cf61fc47972fb7e89be65d1a83332e
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables193
Number of clauses396
Sum of the clauses size1188
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3396
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386137-1305581100/watcher-3386137-1305581100 -o /tmp/evaluation-result-3386137-1305581100/solver-3386137-1305581100 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3386137-1305581100.cnf -threads=4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.63 7.76 7.98 6/530 12585
/proc/meminfo: memFree=12534164/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=49212 CPUtime=0.02 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 458 0 0 0 2 0 0 0 25 0 4 0 669708773 50393088 395 33554432000 4194304 4625164 140736650617120 18446744073709551615 47577714567084 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12585/statm: 12303 395 300 106 0 7806 0
[pid=12585/tid=12586] ppid=12583 vsize=180432 CPUtime=0 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 28 0 0 0 0 0 0 0 25 0 4 0 669708773 184762368 509 33554432000 4194304 4625164 140736650617120 18446744073709551615 4464383 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=180432 CPUtime=0.01 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 29 0 0 0 1 0 0 0 25 0 4 0 669708773 184762368 509 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=180432 CPUtime=0.01 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 55 0 0 0 1 0 0 0 25 0 4 0 669708773 184762368 509 33554432000 4194304 4625164 140736650617120 18446744073709551615 47577714567718 0 0 4096 2 0 0 0 -1 5 0 0 0

[startup+0.0631741 s]
/proc/loadavg: 6.63 7.76 7.98 6/530 12585
/proc/meminfo: memFree=12534164/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=246956 CPUtime=0.21 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 1295 0 0 0 21 0 0 0 25 0 4 0 669708773 252882944 1089 33554432000 4194304 4625164 140736650617120 18446744073709551615 268272732343 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 61739 1089 343 106 0 57242 0
[pid=12585/tid=12586] ppid=12583 vsize=246956 CPUtime=0.04 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 187 0 0 0 4 0 0 0 25 0 4 0 669708773 252882944 1089 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351897 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=246956 CPUtime=0.05 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 192 0 0 0 5 0 0 0 25 0 4 0 669708773 252882944 1089 33554432000 4194304 4625164 140736650617120 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=246956 CPUtime=0.05 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 267 0 0 0 5 0 0 0 25 0 4 0 669708773 252882944 1089 33554432000 4194304 4625164 140736650617120 18446744073709551615 4555467 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 246956

[startup+0.120039 s]
/proc/loadavg: 6.63 7.76 7.98 6/530 12585
/proc/meminfo: memFree=12534164/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=247904 CPUtime=0.44 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 2007 0 0 0 44 0 0 0 25 0 4 0 669708773 253853696 1621 33554432000 4194304 4625164 140736650617120 18446744073709551615 4512818 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 61976 1621 343 106 0 57479 0
[pid=12585/tid=12586] ppid=12583 vsize=247904 CPUtime=0.1 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 373 0 0 0 10 0 0 0 25 0 4 0 669708773 253853696 1621 33554432000 4194304 4625164 140736650617120 18446744073709551615 4352032 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=247904 CPUtime=0.11 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 389 0 0 0 11 0 0 0 25 0 4 0 669708773 253853696 1621 33554432000 4194304 4625164 140736650617120 18446744073709551615 4288602 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=247904 CPUtime=0.11 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 451 0 0 0 11 0 0 0 25 0 4 0 669708773 253853696 1621 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351792 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 247904

[startup+0.30101 s]
/proc/loadavg: 6.63 7.76 7.98 6/530 12585
/proc/meminfo: memFree=12534164/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=250592 CPUtime=1.16 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 4107 0 0 0 116 0 0 0 25 0 4 0 669708773 256606208 3012 33554432000 4194304 4625164 140736650617120 18446744073709551615 4422707 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 62648 3012 343 106 0 58151 0
[pid=12585/tid=12586] ppid=12583 vsize=250592 CPUtime=0.28 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 983 0 0 0 28 0 0 0 25 0 4 0 669708773 256606208 3012 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=250592 CPUtime=0.29 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 989 0 0 0 29 0 0 0 25 0 4 0 669708773 256606208 3012 33554432000 4194304 4625164 140736650617120 18446744073709551615 4464333 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=250592 CPUtime=0.29 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 993 0 0 0 29 0 0 0 25 0 4 0 669708773 256606208 3012 33554432000 4194304 4625164 140736650617120 18446744073709551615 4547221 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1.16
Current children cumulated vsize (KiB) 250592

[startup+0.70094 s]
/proc/loadavg: 6.63 7.76 7.98 6/530 12585
/proc/meminfo: memFree=12534164/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=254608 CPUtime=2.76 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 7659 0 0 0 276 0 0 0 25 0 4 0 669708773 260718592 5042 33554432000 4194304 4625164 140736650617120 18446744073709551615 4329208 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 63652 5042 344 106 0 59155 0
[pid=12585/tid=12586] ppid=12583 vsize=254608 CPUtime=0.68 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 1807 0 0 0 68 0 0 0 25 0 4 0 669708773 260718592 5042 33554432000 4194304 4625164 140736650617120 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=254608 CPUtime=0.69 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 2183 0 0 0 69 0 0 0 25 0 4 0 669708773 260718592 5042 33554432000 4194304 4625164 140736650617120 18446744073709551615 4352691 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=254608 CPUtime=0.69 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 2019 0 0 0 69 0 0 0 25 0 4 0 669708773 260718592 5042 33554432000 4194304 4625164 140736650617120 18446744073709551615 4568384 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2.76
Current children cumulated vsize (KiB) 254608

[startup+1.5008 s]
/proc/loadavg: 6.63 7.76 7.98 10/534 12589
/proc/meminfo: memFree=12492492/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=261500 CPUtime=5.94 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 16189 0 0 0 592 2 0 0 25 0 4 0 669708773 267776000 7890 33554432000 4194304 4625164 140736650617120 18446744073709551615 4396709 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 65375 7890 344 106 0 60878 0
[pid=12585/tid=12586] ppid=12583 vsize=261500 CPUtime=1.48 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 3642 0 0 0 148 0 0 0 25 0 4 0 669708773 267776000 7890 33554432000 4194304 4625164 140736650617120 18446744073709551615 4352035 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=261500 CPUtime=1.48 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 3515 0 0 0 148 0 0 0 25 0 4 0 669708773 267776000 7890 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351811 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=261500 CPUtime=1.48 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 6279 0 0 0 148 0 0 0 25 0 4 0 669708773 267776000 7890 33554432000 4194304 4625164 140736650617120 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 5.94
Current children cumulated vsize (KiB) 261500

[startup+3.10051 s]
/proc/loadavg: 6.63 7.76 7.98 10/534 12589
/proc/meminfo: memFree=12467936/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=268864 CPUtime=12.32 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 31598 0 0 0 1228 4 0 0 25 0 4 0 669708773 275316736 12220 33554432000 4194304 4625164 140736650617120 18446744073709551615 4396763 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 67216 12221 346 106 0 62719 0
[pid=12585/tid=12586] ppid=12583 vsize=268864 CPUtime=3.08 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 5765 0 0 0 307 1 0 0 25 0 4 0 669708773 275316736 12221 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=268864 CPUtime=3.08 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 5483 0 0 0 308 0 0 0 25 0 4 0 669708773 275316736 12221 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=268864 CPUtime=3.09 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 16983 0 0 0 307 2 0 0 25 0 4 0 669708773 275316736 12221 33554432000 4194304 4625164 140736650617120 18446744073709551615 4339865 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 12.32
Current children cumulated vsize (KiB) 268864

[startup+6.30095 s]
/proc/loadavg: 6.74 7.77 7.98 10/534 12589
/proc/meminfo: memFree=12417964/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=278364 CPUtime=25.06 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 46116 0 0 0 2499 7 0 0 25 0 4 0 669708773 285044736 18422 33554432000 4194304 4625164 140736650617120 18446744073709551615 4519334 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 69591 18422 346 106 0 65094 0
[pid=12585/tid=12586] ppid=12583 vsize=278364 CPUtime=6.28 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 8729 0 0 0 627 1 0 0 25 0 4 0 669708773 285044736 18422 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=278364 CPUtime=6.28 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 9284 0 0 0 626 2 0 0 25 0 4 0 669708773 285044736 18422 33554432000 4194304 4625164 140736650617120 18446744073709551615 4352653 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=278364 CPUtime=6.28 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 23337 0 0 0 626 2 0 0 25 0 4 0 669708773 285044736 18422 33554432000 4194304 4625164 140736650617120 18446744073709551615 4552399 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 25.06
Current children cumulated vsize (KiB) 278364

[startup+12.7008 s]
/proc/loadavg: 6.84 7.77 7.98 10/534 12589
/proc/meminfo: memFree=12357128/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=297208 CPUtime=50.56 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 78157 0 0 0 5042 14 0 0 25 0 4 0 669708773 304340992 28989 33554432000 4194304 4625164 140736650617120 18446744073709551615 4519414 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 74302 28989 491 106 0 69805 0
[pid=12585/tid=12586] ppid=12583 vsize=297208 CPUtime=12.68 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 12916 0 0 0 1265 3 0 0 25 0 4 0 669708773 304340992 28989 33554432000 4194304 4625164 140736650617120 18446744073709551615 4352739 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=297208 CPUtime=12.68 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 13969 0 0 0 1264 4 0 0 25 0 4 0 669708773 304340992 28989 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=297208 CPUtime=12.68 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 41807 0 0 0 1264 4 0 0 25 0 4 0 669708773 304340992 28989 33554432000 4194304 4625164 140736650617120 18446744073709551615 4548692 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 50.56
Current children cumulated vsize (KiB) 297208

[startup+25.5005 s]
/proc/loadavg: 7.10 7.78 7.98 10/534 12589
/proc/meminfo: memFree=12275280/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=308084 CPUtime=101.52 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 118877 0 0 0 10129 23 0 0 25 0 4 0 669708773 315478016 41283 33554432000 4194304 4625164 140736650617120 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 77021 41283 367 106 0 72524 0
[pid=12585/tid=12586] ppid=12583 vsize=308084 CPUtime=25.47 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 19937 0 0 0 2542 5 0 0 25 0 4 0 669708773 315478016 41283 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=308084 CPUtime=25.49 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 19160 0 0 0 2543 6 0 0 25 0 4 0 669708773 315478016 41283 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=308084 CPUtime=25.49 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 65343 0 0 0 2540 9 0 0 25 0 4 0 669708773 315478016 41283 33554432000 4194304 4625164 140736650617120 18446744073709551615 4352035 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 101.52
Current children cumulated vsize (KiB) 308084

[startup+51.101 s]
/proc/loadavg: 7.54 7.83 7.99 9/534 12590
/proc/meminfo: memFree=12154980/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=362792 CPUtime=203.49 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 147246 0 0 0 20316 33 0 0 25 0 4 0 669708773 371499008 60584 33554432000 4194304 4625164 140736650617120 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 90698 60584 726 106 0 86201 0
[pid=12585/tid=12586] ppid=12583 vsize=362792 CPUtime=51.08 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 29801 0 0 0 5096 12 0 0 25 0 4 0 669708773 371499008 60584 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=362792 CPUtime=51.08 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 26460 0 0 0 5100 8 0 0 25 0 4 0 669708773 371499008 60584 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=362792 CPUtime=51.08 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 69500 0 0 0 5099 9 0 0 25 0 4 0 669708773 371499008 60584 33554432000 4194304 4625164 140736650617120 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 203.49
Current children cumulated vsize (KiB) 362792

[startup+102.32 s]
/proc/loadavg: 7.87 7.87 7.99 10/534 12591
/proc/meminfo: memFree=11986320/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=539068 CPUtime=407.48 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 183073 0 0 0 40698 50 0 0 25 0 4 0 669708773 552005632 85249 33554432000 4194304 4625164 140736650617120 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 134767 85249 384 106 0 130270 0
[pid=12585/tid=12586] ppid=12583 vsize=539068 CPUtime=102.3 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 37934 0 0 0 10208 22 0 0 25 0 4 0 669708773 552005632 85249 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351901 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=539068 CPUtime=102.3 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 37800 0 0 0 10219 11 0 0 25 0 4 0 669708773 552005632 85249 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=539068 CPUtime=102.3 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 75246 0 0 0 10220 10 0 0 25 0 4 0 669708773 552005632 85249 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 407.48
Current children cumulated vsize (KiB) 539068

[startup+162.301 s]

################
# More data... #
################

[pid=12585] ppid=12583 vsize=1208144 CPUtime=2797 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 2925727 0 0 0 279127 573 0 0 25 0 4 0 669708773 1237139456 230784 33554432000 4194304 4625164 140736650617120 18446744073709551615 4511973 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 302036 230784 347 106 0 297539 0
[pid=12585/tid=12586] ppid=12583 vsize=1208144 CPUtime=702.3 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 155912 0 0 0 70151 79 0 0 25 0 4 0 669708773 1237139456 230784 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=1208144 CPUtime=702.3 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 126239 0 0 0 70180 50 0 0 25 0 4 0 669708773 1237139456 230784 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351975 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=1208144 CPUtime=702.3 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 1664204 0 0 0 69970 260 0 0 25 0 4 0 669708773 1237139456 230784 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2797
Current children cumulated vsize (KiB) 1208144

[startup+762.32 s]
/proc/loadavg: 8.10 8.09 8.03 9/534 12699
/proc/meminfo: memFree=10847628/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=1218572 CPUtime=3036.02 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 3317576 0 0 0 302960 642 0 0 25 0 4 0 669708773 1247817728 239978 33554432000 4194304 4625164 140736650617120 18446744073709551615 4329151 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 304643 239978 358 106 0 300146 0
[pid=12585/tid=12586] ppid=12583 vsize=1218572 CPUtime=762.31 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 159992 0 0 0 76147 84 0 0 25 0 4 0 669708773 1247817728 239978 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=1218572 CPUtime=762.32 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 129467 0 0 0 76179 53 0 0 25 0 4 0 669708773 1247817728 239978 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=1218572 CPUtime=762.32 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 1891752 0 0 0 75940 292 0 0 25 0 4 0 669708773 1247817728 239978 33554432000 4194304 4625164 140736650617120 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3036.02
Current children cumulated vsize (KiB) 1218572

[startup+822.301 s]
/proc/loadavg: 8.33 8.15 8.05 9/534 12701
/proc/meminfo: memFree=10705616/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=1323424 CPUtime=3274.91 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 3597492 0 0 0 326802 689 0 0 25 0 4 0 669708773 1355186176 261466 33554432000 4194304 4625164 140736650617120 18446744073709551615 4393091 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 330856 261466 392 106 0 326359 0
[pid=12585/tid=12586] ppid=12583 vsize=1323424 CPUtime=822.29 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 166666 0 0 0 82144 85 0 0 25 0 4 0 669708773 1355186176 261466 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351798 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=1323424 CPUtime=822.3 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 140901 0 0 0 82174 56 0 0 25 0 4 0 669708773 1355186176 261466 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351984 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=1323424 CPUtime=822.29 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 2131136 0 0 0 81899 330 0 0 25 0 4 0 669708773 1355186176 261466 33554432000 4194304 4625164 140736650617120 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3274.91
Current children cumulated vsize (KiB) 1323424

[startup+882.301 s]
/proc/loadavg: 8.12 8.12 8.05 9/534 12702
/proc/meminfo: memFree=10589632/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=1496952 CPUtime=3513.86 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 3862869 0 0 0 350653 733 0 0 25 0 4 0 669708773 1532878848 273779 33554432000 4194304 4625164 140736650617120 18446744073709551615 4329151 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 374238 273779 395 106 0 369741 0
[pid=12585/tid=12586] ppid=12583 vsize=1496952 CPUtime=882.3 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 167423 0 0 0 88144 86 0 0 25 0 4 0 669708773 1532878848 273779 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=1496952 CPUtime=882.3 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 179099 0 0 0 88167 63 0 0 25 0 4 0 669708773 1532878848 273779 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=1496952 CPUtime=882.3 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 2338711 0 0 0 87866 364 0 0 25 0 4 0 669708773 1532878848 273779 33554432000 4194304 4625164 140736650617120 18446744073709551615 4552423 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3513.86
Current children cumulated vsize (KiB) 1496952

[startup+942.3 s]
/proc/loadavg: 8.18 8.13 8.05 9/534 12704
/proc/meminfo: memFree=10510968/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=1507776 CPUtime=3752.82 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 4108032 0 0 0 374504 778 0 0 25 0 4 0 669708773 1543962624 281461 33554432000 4194304 4625164 140736650617120 18446744073709551615 4392981 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 376944 281461 382 106 0 372447 0
[pid=12585/tid=12586] ppid=12583 vsize=1507776 CPUtime=942.29 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 169739 0 0 0 94141 88 0 0 25 0 4 0 669708773 1543962624 281461 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=1507776 CPUtime=942.3 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 183212 0 0 0 94165 65 0 0 25 0 4 0 669708773 1543962624 281461 33554432000 4194304 4625164 140736650617120 18446744073709551615 4352053 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=1507776 CPUtime=942.3 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 2576631 0 0 0 93827 403 0 0 25 0 4 0 669708773 1543962624 281461 33554432000 4194304 4625164 140736650617120 18446744073709551615 4552315 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3752.82
Current children cumulated vsize (KiB) 1507776

[startup+1002.3 s]
/proc/loadavg: 8.12 8.12 8.05 9/534 12705
/proc/meminfo: memFree=10423628/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=1518468 CPUtime=3991.77 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 4311889 0 0 0 398365 812 0 0 25 0 4 0 669708773 1554911232 288490 33554432000 4194304 4625164 140736650617120 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 379617 288490 344 106 0 375120 0
[pid=12585/tid=12586] ppid=12583 vsize=1518468 CPUtime=1002.3 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 172275 0 0 0 100138 92 0 0 25 0 4 0 669708773 1554911232 288490 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=1518468 CPUtime=1002.31 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 186789 0 0 0 100164 67 0 0 25 0 4 0 669708773 1554911232 288490 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351811 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=1518468 CPUtime=1002.31 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 2773472 0 0 0 99798 433 0 0 25 0 4 0 669708773 1554911232 288490 33554432000 4194304 4625164 140736650617120 18446744073709551615 4547918 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3991.77
Current children cumulated vsize (KiB) 1518468

[startup+1062.3 s]
/proc/loadavg: 8.16 8.12 8.06 9/534 12707
/proc/meminfo: memFree=10351176/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=1528632 CPUtime=4230.74 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 4542051 0 0 0 422220 854 0 0 25 0 4 0 669708773 1565319168 300007 33554432000 4194304 4625164 140736650617120 18446744073709551615 4512156 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 382158 300007 387 106 0 377661 0
[pid=12585/tid=12586] ppid=12583 vsize=1528632 CPUtime=1062.3 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 176775 0 0 0 106133 97 0 0 25 0 4 0 669708773 1565319168 300007 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=1528632 CPUtime=1062.31 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 191191 0 0 0 106161 70 0 0 25 0 4 0 669708773 1565319168 300007 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=1528632 CPUtime=1062.3 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 2993995 0 0 0 105764 466 0 0 25 0 4 0 669708773 1565319168 300007 33554432000 4194304 4625164 140736650617120 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4230.74
Current children cumulated vsize (KiB) 1528632

[startup+1122.3 s]
/proc/loadavg: 8.11 8.11 8.06 9/534 12709
/proc/meminfo: memFree=10279472/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=1537608 CPUtime=4469.7 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 4780565 0 0 0 446072 898 0 0 25 0 4 0 669708773 1574510592 310797 33554432000 4194304 4625164 140736650617120 18446744073709551615 4392976 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 384402 310797 368 106 0 379905 0
[pid=12585/tid=12586] ppid=12583 vsize=1537608 CPUtime=1122.3 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 183437 0 0 0 112127 103 0 0 25 0 4 0 669708773 1574510592 310797 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=1537608 CPUtime=1122.31 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 193753 0 0 0 112159 72 0 0 25 0 4 0 669708773 1574510592 310797 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=1537608 CPUtime=1122.3 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 3222582 0 0 0 111729 501 0 0 25 0 4 0 669708773 1574510592 310797 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4469.7
Current children cumulated vsize (KiB) 1537608

[startup+1182.3 s]
/proc/loadavg: 8.14 8.12 8.06 9/534 12710
/proc/meminfo: memFree=10236528/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=1574704 CPUtime=4708.66 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 5035822 0 0 0 469923 943 0 0 25 0 4 0 669708773 1612496896 315557 33554432000 4194304 4625164 140736650617120 18446744073709551615 4512818 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 393676 315558 352 106 0 389179 0
[pid=12585/tid=12586] ppid=12583 vsize=1574704 CPUtime=1182.3 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 185035 0 0 0 118124 106 0 0 25 0 4 0 669708773 1612496896 315558 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=1574704 CPUtime=1182.31 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 196268 0 0 0 118158 73 0 0 25 0 4 0 669708773 1612496896 315558 33554432000 4194304 4625164 140736650617120 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=1574704 CPUtime=1182.31 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 3458531 0 0 0 117691 540 0 0 25 0 4 0 669708773 1612496896 315558 33554432000 4194304 4625164 140736650617120 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4708.66
Current children cumulated vsize (KiB) 1574704



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1205.3 s]
/proc/loadavg: 8.24 8.14 8.07 9/534 12711
/proc/meminfo: memFree=10189396/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=1721184 CPUtime=4800.3 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 5152712 0 0 0 479068 962 0 0 25 0 4 0 669708773 1762492416 324035 33554432000 4194304 4625164 140736650617120 18446744073709551615 4512818 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 430296 324035 382 106 0 425799 0
[pid=12585/tid=12586] ppid=12583 vsize=1721184 CPUtime=1205.33 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 228479 0 0 0 120420 113 0 0 25 0 4 0 669708773 1762492416 324036 33554432000 4194304 4625164 140736650617120 18446744073709551615 4352859 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=1721184 CPUtime=1205.33 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 197268 0 0 0 120459 74 0 0 25 0 4 0 669708773 1762492416 324036 33554432000 4194304 4625164 140736650617120 18446744073709551615 4352157 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=1721184 CPUtime=1205.32 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 3530754 0 0 0 119981 551 0 0 25 0 4 0 669708773 1762492416 324036 33554432000 4194304 4625164 140736650617120 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.3
Current children cumulated vsize (KiB) 1721184

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1205.3 s]
/proc/loadavg: 8.24 8.14 8.07 9/534 12711
/proc/meminfo: memFree=10189396/32950928 swapFree=60199076/67111528
[pid=12585] ppid=12583 vsize=1721184 CPUtime=4800.3 cores=1,3,5,7
/proc/12585/stat : 12585 (SArTagnan) R 12583 12585 12217 0 -1 4202496 5152712 0 0 0 479068 962 0 0 25 0 4 0 669708773 1762492416 324035 33554432000 4194304 4625164 140736650617120 18446744073709551615 4512818 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12585/statm: 430296 324035 382 106 0 425799 0
[pid=12585/tid=12586] ppid=12583 vsize=1721184 CPUtime=1205.33 cores=1,3,5,7
/proc/12585/task/12586/stat : 12586 (SArTagnan) R 12583 12585 12217 0 -1 4202560 228479 0 0 0 120420 113 0 0 25 0 4 0 669708773 1762492416 324036 33554432000 4194304 4625164 140736650617120 18446744073709551615 4352859 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12585/tid=12587] ppid=12583 vsize=1721184 CPUtime=1205.33 cores=1,3,5,7
/proc/12585/task/12587/stat : 12587 (SArTagnan) R 12583 12585 12217 0 -1 4202560 197268 0 0 0 120459 74 0 0 25 0 4 0 669708773 1762492416 324036 33554432000 4194304 4625164 140736650617120 18446744073709551615 4352157 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12585/tid=12588] ppid=12583 vsize=1721184 CPUtime=1205.32 cores=1,3,5,7
/proc/12585/task/12588/stat : 12588 (SArTagnan) R 12583 12585 12217 0 -1 4202560 3530754 0 0 0 119981 551 0 0 25 0 4 0 669708773 1762492416 324036 33554432000 4194304 4625164 140736650617120 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.3
Current children cumulated vsize (KiB) 1721184

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1205.37
CPU time (s): 4800.39
CPU user time (s): 4790.72
CPU system time (s): 9.66853
CPU usage (%): 398.249
Max. virtual memory (cumulated for all children) (KiB): 1875208

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4790.72
system time used= 9.66853
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 5152713
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 991
involuntary context switches= 29786

runsolver used 5.9451 second user time and 15.0947 second system time

The end

Launcher Data 

Begin job on node122 at 2011-05-16 23:25:00
IDJOB=3386137
IDBENCH=83156
IDSOLVER=1871
FILE ID=node122/3386137-1305581100
RUNJOBID= node122-1305581100-12563
PBS_JOBID= 13350225
Free space on /tmp= 73380 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT11/crafted/spence/sgen/sgen3-n190-s29985700-unsat.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386137-1305581100/watcher-3386137-1305581100 -o /tmp/evaluation-result-3386137-1305581100/solver-3386137-1305581100 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3386137-1305581100.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f2cf61fc47972fb7e89be65d1a83332e
RANDOM SEED=1112270411

node122.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      12534444 kB
Buffers:        333300 kB
Cached:        4383832 kB
SwapCached:      50104 kB
Active:       15657320 kB
Inactive:      4466028 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      12534444 kB
SwapTotal:    67111528 kB
SwapFree:     60199076 kB
Dirty:             872 kB
Writeback:          16 kB
AnonPages:    15366472 kB
Mapped:          31368 kB
Slab:           137480 kB
PageTables:      86408 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 160611380 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73380 MiB
End job on node122 at 2011-05-16 23:45:07