Trace number 3484404

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-random (fixed, 1200 s timeout)? (MO) 700.98 700.662

General information on the benchmark

Namerandom/medium/
unif-k7-r89-v80-c7120-S1637697223-097.UNKNOWN.cnf
MD5SUMa7e7ad4822818289e739fae39e69d3b8
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables80
Number of clauses7120
Sum of the clauses size49840
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 57120

Solver Data 

0.33/0.50	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.33/0.50	c INFO: solving HOME/instance-3484404-1307749518.cnf
0.33/0.50	c INFO: computing task features
0.33/0.55	c INFO: computed variable-clause graph statistics
0.33/0.55	c INFO: computed clause balance statistics
0.63/0.81	c INFO: computed variable balance statistics
0.73/0.92	c INFO: computed small-clause counts
0.73/0.92	c INFO: computed Horn-clause counts
0.73/0.93	c INFO: computed variable graph statistics
0.73/0.93	c INFO: collected features for HOME/instance-3484404-1307749518.cnf in 0.42 s
0.73/0.93	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.49)
100.94/101.07	c INFO: running gnovelty+2@0 for 97 with 1099 remaining (b = 0.10)
201.14/201.22	c INFO: running adaptg2wsat2009++@0 for 97 with 999 remaining (b = 0.06)
301.22/301.35	c INFO: running adaptg2wsat2009++@0 for 97 with 899 remaining (b = 0.04)
401.43/401.49	c INFO: running adaptg2wsat2009++@0 for 97 with 799 remaining (b = 0.03)
501.73/501.63	c INFO: running gnovelty+2@0 for 97 with 699 remaining (b = 0.03)
600.94/600.79	c INFO: running FH@0 for 97 with 600 remaining (b = 0.02)
700.29/699.94	c INFO: running adaptg2wsat2009++@0 for 97 with 500 remaining (b = 0.02)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 988)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3484404-1307749518/watcher-3484404-1307749518 -o /tmp/evaluation-result-3484404-1307749518/solver-3484404-1307749518 -C 1200 -W 1300 -M 7680 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3484404-1307749518.cnf -seed 770318917 -budget 1200 

running on 2 cores: 4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1200 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1230 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.45 4.16 3.87 4/204 13969
/proc/meminfo: memFree=31388796/32951132 swapFree=67052544/67111528
[pid=13969] ppid=13967 vsize=11804 CPUtime=0 cores=4,6
/proc/13969/stat : 13969 (runsolver) S 13967 13969 12804 0 -1 4202560 42 0 0 0 0 0 0 0 18 0 1 0 886550457 12087296 50 33554432000 4194304 5436572 140736438144672 18446744073709551615 4883175 0 2147483391 4096 24578 18446744071699870519 0 0 17 4 0 0 0
/proc/13969/statm: 2951 50 31 304 0 2643 0

[startup+0.0135071 s]
/proc/loadavg: 4.45 4.16 3.87 4/204 13969
/proc/meminfo: memFree=31388796/32951132 swapFree=67052544/67111528
[pid=13969] ppid=13967 vsize=11804 CPUtime=0 cores=4,6
/proc/13969/stat : 13969 (runsolver) S 13967 13969 12804 0 -1 4202560 42 0 0 0 0 0 0 0 18 0 1 0 886550457 12087296 50 33554432000 4194304 5436572 140736438144672 18446744073709551615 4883175 0 2147483391 4096 24578 18446744071699870519 0 0 17 4 0 0 0
/proc/13969/statm: 2951 50 31 304 0 2643 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 11804

[startup+0.100631 s]
/proc/loadavg: 4.45 4.16 3.87 4/204 13969
/proc/meminfo: memFree=31388796/32951132 swapFree=67052544/67111528
[pid=13969] ppid=13967 vsize=43624 CPUtime=0.04 cores=4,6
/proc/13969/stat : 13969 (python) S 13967 13969 12804 0 -1 4202496 1881 0 0 0 3 1 0 0 17 0 1 0 886550457 44670976 1654 33554432000 4194304 5683644 140737359489440 18446744073709551615 242970548224 0 2147483391 16781312 2 18446744071699109054 0 0 17 6 0 0 0
/proc/13969/statm: 10906 1654 467 364 0 1171 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 43624

[startup+0.300952 s]
/proc/loadavg: 4.45 4.16 3.87 4/204 13969
/proc/meminfo: memFree=31388796/32951132 swapFree=67052544/67111528
[pid=13969] ppid=13967 vsize=135752 CPUtime=0.19 cores=4,6
/proc/13969/stat : 13969 (python) S 13967 13969 12804 0 -1 4202496 6017 931 1 0 15 4 0 0 17 0 1 0 886550457 139010048 4170 33554432000 4194304 5683644 140737359489440 18446744073709551615 242970548224 0 2147483391 16781312 2 18446744071699109054 0 0 17 6 0 0 0
/proc/13969/statm: 33938 4170 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135752

[startup+0.700388 s]
/proc/loadavg: 4.45 4.16 3.87 4/204 13969
/proc/meminfo: memFree=31388796/32951132 swapFree=67052544/67111528
[pid=13969] ppid=13967 vsize=279952 CPUtime=0.53 cores=4,6
/proc/13969/stat : 13969 (python) R 13967 13969 12804 0 -1 4202496 9816 931 1 0 47 6 0 0 17 0 1 0 886550457 286670848 7372 33554432000 4194304 5683644 140737359489440 18446744073709551615 4540741 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/13969/statm: 69988 7372 1826 364 0 5912 0
Current children cumulated CPU time (s) 0.53
Current children cumulated vsize (KiB) 279952

[startup+1.50031 s]
/proc/loadavg: 4.45 4.16 3.87 5/208 13977
/proc/meminfo: memFree=31363064/32951132 swapFree=67052544/67111528
[pid=13969] ppid=13967 vsize=290220 CPUtime=0.77 cores=4,6
/proc/13969/stat : 13969 (python) S 13967 13969 12804 0 -1 4202496 10106 931 1 0 71 6 0 0 20 0 2 0 886550457 297185280 7419 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/13969/statm: 72555 7419 1869 364 0 8473 0
[pid=13975] ppid=13969 vsize=279976 CPUtime=0 cores=4,6
/proc/13975/stat : 13975 (python) S 13969 13969 12804 0 -1 4202560 732 0 0 0 0 0 0 0 20 0 1 0 886550550 286695424 5900 33554432000 4194304 5683644 140737359489440 18446744073709551615 242970570255 0 0 16781312 514 3742246378424762367 0 0 17 6 0 0 0
/proc/13975/statm: 69994 5900 354 364 0 5912 0
[pid=13977] ppid=13975 vsize=1018084 CPUtime=0.55 cores=4,6
/proc/13977/stat : 13977 (gnovelty+2) R 13975 13977 13977 0 -1 4202496 551 0 0 0 55 0 0 0 23 0 1 0 886550551 1042518016 287 33554432000 134512640 135663704 4286850304 18446744073709551615 134524534 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/13977/statm: 254521 287 113 282 0 254237 0
[pid=13969/tid=13976] ppid=13967 vsize=290220 CPUtime=0 cores=4,6
/proc/13969/task/13976/stat : 13976 (python) S 13967 13969 12804 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 2 0 886550551 297185280 7419 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1.32
Current children cumulated vsize (KiB) 1588280

[startup+3.10101 s]
/proc/loadavg: 4.45 4.16 3.87 5/208 13977
/proc/meminfo: memFree=31362444/32951132 swapFree=67052544/67111528
[pid=13969] ppid=13967 vsize=290220 CPUtime=0.77 cores=4,6
/proc/13969/stat : 13969 (python) S 13967 13969 12804 0 -1 4202496 10106 931 1 0 71 6 0 0 20 0 2 0 886550457 297185280 7419 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/13969/statm: 72555 7419 1869 364 0 8473 0
[pid=13975] ppid=13969 vsize=279976 CPUtime=0 cores=4,6
/proc/13975/stat : 13975 (python) S 13969 13969 12804 0 -1 4202560 732 0 0 0 0 0 0 0 18 0 1 0 886550550 286695424 5900 33554432000 4194304 5683644 140737359489440 18446744073709551615 242970570255 0 0 16781312 514 3742246378424729859 0 0 17 6 0 0 0
/proc/13975/statm: 69994 5900 354 364 0 5912 0
[pid=13977] ppid=13975 vsize=1018084 CPUtime=2.15 cores=4,6
/proc/13977/stat : 13977 (gnovelty+2) R 13975 13977 13977 0 -1 4202496 551 0 0 0 215 0 0 0 25 0 1 0 886550551 1042518016 287 33554432000 134512640 135663704 4286850304 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/13977/statm: 254521 287 113 282 0 254237 0
[pid=13969/tid=13976] ppid=13967 vsize=290220 CPUtime=0 cores=4,6
/proc/13969/task/13976/stat : 13976 (python) S 13967 13969 12804 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 2 0 886550551 297185280 7419 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2.92
Current children cumulated vsize (KiB) 1588280

[startup+6.30048 s]
/proc/loadavg: 4.49 4.18 3.88 5/208 13977
/proc/meminfo: memFree=31360964/32951132 swapFree=67052544/67111528
[pid=13969] ppid=13967 vsize=290220 CPUtime=0.77 cores=4,6
/proc/13969/stat : 13969 (python) S 13967 13969 12804 0 -1 4202496 10106 931 1 0 71 6 0 0 20 0 2 0 886550457 297185280 7419 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/13969/statm: 72555 7419 1869 364 0 8473 0
[pid=13975] ppid=13969 vsize=279976 CPUtime=0 cores=4,6
/proc/13975/stat : 13975 (python) S 13969 13969 12804 0 -1 4202560 866 0 0 0 0 0 0 0 18 0 1 0 886550550 286695424 5923 33554432000 4194304 5683644 140737359489440 18446744073709551615 242970570255 0 0 16781312 514 3742246378424729859 0 0 17 6 0 0 0
/proc/13975/statm: 69994 5923 377 364 0 5912 0
[pid=13977] ppid=13975 vsize=1018084 CPUtime=5.35 cores=4,6
/proc/13977/stat : 13977 (gnovelty+2) R 13975 13977 13977 0 -1 4202496 551 0 0 0 535 0 0 0 25 0 1 0 886550551 1042518016 287 33554432000 134512640 135663704 4286850304 18446744073709551615 134525213 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/13977/statm: 254521 287 113 282 0 254237 0
[pid=13969/tid=13976] ppid=13967 vsize=290220 CPUtime=0 cores=4,6
/proc/13969/task/13976/stat : 13976 (python) S 13967 13969 12804 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 2 0 886550551 297185280 7419 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 6.12
Current children cumulated vsize (KiB) 1588280

[startup+12.7004 s]
/proc/loadavg: 4.61 4.21 3.89 5/208 13977
/proc/meminfo: memFree=31360240/32951132 swapFree=67052544/67111528
[pid=13969] ppid=13967 vsize=290220 CPUtime=0.77 cores=4,6
/proc/13969/stat : 13969 (python) S 13967 13969 12804 0 -1 4202496 10106 931 1 0 71 6 0 0 20 0 2 0 886550457 297185280 7419 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/13969/statm: 72555 7419 1869 364 0 8473 0
[pid=13975] ppid=13969 vsize=279976 CPUtime=0 cores=4,6
/proc/13975/stat : 13975 (python) S 13969 13969 12804 0 -1 4202560 868 0 0 0 0 0 0 0 18 0 1 0 886550550 286695424 5923 33554432000 4194304 5683644 140737359489440 18446744073709551615 242970570255 0 0 16781312 514 4362862138982659 0 0 17 6 0 0 0
/proc/13975/statm: 69994 5923 377 364 0 5912 0
[pid=13977] ppid=13975 vsize=1018084 CPUtime=11.75 cores=4,6
/proc/13977/stat : 13977 (gnovelty+2) R 13975 13977 13977 0 -1 4202496 551 0 0 0 1175 0 0 0 25 0 1 0 886550551 1042518016 287 33554432000 134512640 135663704 4286850304 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/13977/statm: 254521 287 113 282 0 254237 0
[pid=13969/tid=13976] ppid=13967 vsize=290220 CPUtime=0 cores=4,6
/proc/13969/task/13976/stat : 13976 (python) S 13967 13969 12804 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 2 0 886550551 297185280 7419 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 12.52
Current children cumulated vsize (KiB) 1588280

[startup+25.5012 s]
/proc/loadavg: 4.68 4.25 3.91 5/208 13977
/proc/meminfo: memFree=31359856/32951132 swapFree=67052544/67111528
[pid=13969] ppid=13967 vsize=290220 CPUtime=0.77 cores=4,6
/proc/13969/stat : 13969 (python) S 13967 13969 12804 0 -1 4202496 10106 931 1 0 71 6 0 0 20 0 2 0 886550457 297185280 7419 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/13969/statm: 72555 7419 1869 364 0 8473 0
[pid=13975] ppid=13969 vsize=279976 CPUtime=0.02 cores=4,6
/proc/13975/stat : 13975 (python) S 13969 13969 12804 0 -1 4202560 873 0 0 0 1 1 0 0 18 0 1 0 886550550 286695424 5923 33554432000 4194304 5683644 140737359489440 18446744073709551615 242970570255 0 0 16781312 514 4362862138982659 0 0 17 6 0 0 0
/proc/13975/statm: 69994 5923 377 364 0 5912 0
[pid=13977] ppid=13975 vsize=1018084 CPUtime=24.55 cores=4,6
/proc/13977/stat : 13977 (gnovelty+2) R 13975 13977 13977 0 -1 4202496 551 0 0 0 2455 0 0 0 25 0 1 0 886550551 1042518016 287 33554432000 134512640 135663704 4286850304 18446744073709551615 134525278 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/13977/statm: 254521 287 113 282 0 254237 0
[pid=13969/tid=13976] ppid=13967 vsize=290220 CPUtime=0 cores=4,6
/proc/13969/task/13976/stat : 13976 (python) S 13967 13969 12804 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 2 0 886550551 297185280 7419 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 25.34
Current children cumulated vsize (KiB) 1588280

[startup+51.1008 s]
/proc/loadavg: 4.53 4.24 3.91 5/208 13977
/proc/meminfo: memFree=31355592/32951132 swapFree=67052544/67111528
[pid=13969] ppid=13967 vsize=290220 CPUtime=0.77 cores=4,6
/proc/13969/stat : 13969 (python) S 13967 13969 12804 0 -1 4202496 10106 931 1 0 71 6 0 0 20 0 2 0 886550457 297185280 7419 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/13969/statm: 72555 7419 1869 364 0 8473 0
[pid=13975] ppid=13969 vsize=279976 CPUtime=0.05 cores=4,6
/proc/13975/stat : 13975 (python) S 13969 13969 12804 0 -1 4202560 886 0 0 0 3 2 0 0 18 0 1 0 886550550 286695424 5923 33554432000 4194304 5683644 140737359489440 18446744073709551615 242970570255 0 0 16781312 514 4362862138982659 0 0 17 6 0 0 0
/proc/13975/statm: 69994 5923 377 364 0 5912 0
[pid=13977] ppid=13975 vsize=1018084 CPUtime=50.15 cores=4,6
/proc/13977/stat : 13977 (gnovelty+2) R 13975 13977 13977 0 -1 4202496 551 0 0 0 5015 0 0 0 25 0 1 0 886550551 1042518016 287 33554432000 134512640 135663704 4286850304 18446744073709551615 134524888 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/13977/statm: 254521 287 113 282 0 254237 0
[pid=13969/tid=13976] ppid=13967 vsize=290220 CPUtime=0 cores=4,6
/proc/13969/task/13976/stat : 13976 (python) S 13967 13969 12804 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 2 0 886550551 297185280 7419 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 50.97
Current children cumulated vsize (KiB) 1588280

[startup+102.307 s]
/proc/loadavg: 4.23 4.20 3.91 5/220 13991
/proc/meminfo: memFree=31331684/32951132 swapFree=67052544/67111528
[pid=13969] ppid=13967 vsize=300476 CPUtime=0.78 cores=4,6
/proc/13969/stat : 13969 (python) S 13967 13969 12804 0 -1 4202496 10958 931 1 0 71 7 0 0 20 0 3 0 886550457 307687424 7426 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/13969/statm: 75119 7426 1872 364 0 11034 0
[pid=13975] ppid=13969 vsize=290220 CPUtime=0.12 cores=4,6
/proc/13975/stat : 13975 (python) S 13969 13969 12804 0 -1 4202560 1139 0 0 0 7 5 0 0 19 0 2 0 886550550 297185280 5966 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/13975/statm: 72555 5966 415 364 0 8473 0
[pid=13977] ppid=13975 vsize=1018084 CPUtime=100.12 cores=4,6
/proc/13977/stat : 13977 (gnovelty+2) T 13975 13977 13977 0 -1 4202496 551 0 0 0 10012 0 0 0 25 0 1 0 886550551 1042518016 287 33554432000 134512640 135663704 4286850304 18446744073709551615 134525258 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/13977/statm: 254521 287 113 282 0 254237 0
[pid=13975/tid=13988] ppid=13969 vsize=290220 CPUtime=0 cores=4,6
/proc/13975/task/13988/stat : 13988 (python) S 13969 13969 12804 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 886560564 297185280 5966 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 514 18446604453784002880 0 0 -1 6 0 0 0
[pid=13989] ppid=13969 vsize=290232 CPUtime=0 cores=4,6
/proc/13989/stat : 13989 (python) S 13969 13969 12804 0 -1 4202560 736 0 0 0 0 0 0 0 18 0 1 0 886560564 297197568 5898 33554432000 4194304 5683644 140737359489440 18446744073709551615 242970570310 0 0 16781312 514 4201097869463093507 0 0 17 4 0 0 0
/proc/13989/statm: 72558 5898 348 364 0 8473 0
[pid=13991] ppid=13989 vsize=1018084 CPUtime=1.22 cores=4,6
/proc/13991/stat : 13991 (gnovelty+2) R 13989 13991 13991 0 -1 4202496 552 0 0 0 122 0 0 0 25 0 1 0 886560565 1042518016 287 33554432000 134512640 135663704 4293521472 18446744073709551615 134525247 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/13991/statm: 254521 287 113 282 0 254237 0
[pid=13969/tid=13976] ppid=13967 vsize=300476 CPUtime=0 cores=4,6
/proc/13969/task/13976/stat : 13976 (python) S 13967 13969 12804 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 3 0 886550551 307687424 7426 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
[pid=13969/tid=13990] ppid=13967 vsize=300476 CPUtime=0 cores=4,6
/proc/13969/task/13990/stat : 13990 (python) S 13967 13969 12804 0 -1 4202560 28 931 0 0 0 0 0 0 20 0 3 0 886560565 307687424 7426 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 102.24
Current children cumulated vsize (KiB) 2917096

[startup+162.301 s]
/proc/loadavg: 4.29 4.24 3.94 5/220 14043
/proc/meminfo: memFree=31319344/32951132 swapFree=67052544/67111528
[pid=13969] ppid=13967 vsize=300476 CPUtime=0.78 cores=4,6
/proc/13969/stat : 13969 (python) S 13967 13969 12804 0 -1 4202496 10958 931 1 0 71 7 0 0 20 0 3 0 886550457 307687424 7426 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/13969/statm: 75119 7426 1872 364 0 11034 0
[pid=13975] ppid=13969 vsize=290220 CPUtime=0.12 cores=4,6

################
# More data... #
################

/proc/13989/statm: 72558 5951 400 364 0 8473 0
[pid=13991] ppid=13989 vsize=1018084 CPUtime=100 cores=4,6
/proc/13991/stat : 13991 (gnovelty+2) T 13989 13991 13991 0 -1 4202496 552 0 0 0 10000 0 0 0 25 0 1 0 886560565 1042518016 287 33554432000 134512640 135663704 4293521472 18446744073709551615 134525493 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/13991/statm: 254521 287 113 282 0 254237 0
[pid=13989/tid=14044] ppid=13969 vsize=290232 CPUtime=0 cores=4,6
/proc/13989/task/14044/stat : 14044 (python) S 13969 13969 12804 0 -1 4202560 51 0 0 0 0 0 0 0 19 0 2 0 886570578 297197568 5951 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 514 18446604453890837568 0 0 -1 6 0 0 0
[pid=14045] ppid=13969 vsize=300488 CPUtime=0.13 cores=4,6
/proc/14045/stat : 14045 (python) S 13969 13969 12804 0 -1 4202560 1085 0 0 0 8 5 0 0 19 0 2 0 886570579 307699712 5955 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/14045/statm: 75122 5955 400 364 0 11034 0
[pid=14047] ppid=14045 vsize=417056 CPUtime=99.98 cores=4,6
/proc/14047/stat : 14047 (adaptg2wsat2009) T 14045 14047 14047 0 -1 4202496 719 0 0 0 9998 0 0 0 25 0 1 0 886570579 427065344 450 33554432000 134512640 135107411 4291031888 18446744073709551615 134531741 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/14047/statm: 104264 450 59 146 0 103140 0
[pid=14045/tid=14194] ppid=13969 vsize=300488 CPUtime=0 cores=4,6
/proc/14045/task/14194/stat : 14194 (python) S 13969 13969 12804 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 886580591 307699712 5955 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 514 18446604453781078208 0 0 -1 6 0 0 0
[pid=14195] ppid=13969 vsize=310744 CPUtime=0.13 cores=4,6
/proc/14195/stat : 14195 (python) S 13969 13969 12804 0 -1 4202560 1097 0 0 0 6 7 0 0 19 0 2 0 886580592 318201856 5959 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/14195/statm: 77686 5959 400 364 0 13595 0
[pid=14197] ppid=14195 vsize=417056 CPUtime=100.11 cores=4,6
/proc/14197/stat : 14197 (adaptg2wsat2009) T 14195 14197 14197 0 -1 4202496 718 0 0 0 10009 2 0 0 25 0 1 0 886580592 427065344 452 33554432000 134512640 135107411 4292083936 18446744073709551615 134531604 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/14197/statm: 104264 452 59 146 0 103140 0
[pid=14195/tid=14254] ppid=13969 vsize=310744 CPUtime=0 cores=4,6
/proc/14195/task/14254/stat : 14254 (python) S 13969 13969 12804 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 886590605 318201856 5959 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 514 18446604453851017536 0 0 -1 6 0 0 0
[pid=14255] ppid=13969 vsize=321000 CPUtime=0.13 cores=4,6
/proc/14255/stat : 14255 (python) S 13969 13969 12804 0 -1 4202560 1088 0 0 0 9 4 0 0 19 0 2 0 886590606 328704000 5963 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/14255/statm: 80250 5963 400 364 0 16156 0
[pid=14257] ppid=14255 vsize=417056 CPUtime=100.12 cores=4,6
/proc/14257/stat : 14257 (adaptg2wsat2009) T 14255 14257 14257 0 -1 4202496 719 0 0 0 10012 0 0 0 25 0 1 0 886590607 427065344 450 33554432000 134512640 135107411 4291250528 18446744073709551615 134532349 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/14257/statm: 104264 450 59 146 0 103140 0
[pid=14255/tid=14264] ppid=13969 vsize=321000 CPUtime=0 cores=4,6
/proc/14255/task/14264/stat : 14264 (python) S 13969 13969 12804 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 886600620 328704000 5963 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 514 18446604443456656704 0 0 -1 6 0 0 0
[pid=14265] ppid=13969 vsize=331256 CPUtime=0.13 cores=4,6
/proc/14265/stat : 14265 (python) S 13969 13969 12804 0 -1 4202560 1084 0 0 0 9 4 0 0 19 0 2 0 886600620 339206144 5967 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/14265/statm: 82814 5967 400 364 0 18717 0
[pid=14267] ppid=14265 vsize=1018084 CPUtime=99.14 cores=4,6
/proc/14267/stat : 14267 (gnovelty+2) T 14265 14267 14267 0 -1 4202496 549 0 0 0 9914 0 0 0 25 0 1 0 886600621 1042518016 287 33554432000 134512640 135663704 4291053248 18446744073709551615 134525684 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/14267/statm: 254521 287 113 282 0 254237 0
[pid=14265/tid=14273] ppid=13969 vsize=331256 CPUtime=0 cores=4,6
/proc/14265/task/14273/stat : 14273 (python) S 13969 13969 12804 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 886610535 339206144 5967 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 514 18446604452741558464 0 0 -1 6 0 0 0
[pid=14274] ppid=13969 vsize=341512 CPUtime=0.05 cores=4,6
/proc/14274/stat : 14274 (python) S 13969 13969 12804 0 -1 4202560 896 0 0 0 3 2 0 0 18 0 1 0 886610536 349708288 5941 33554432000 4194304 5683644 140737359489440 18446744073709551615 242970570310 0 0 16781312 514 4362862138982658 0 0 17 6 0 0 0
/proc/14274/statm: 85378 5941 371 364 0 21278 0
[pid=14276] ppid=14274 vsize=247236 CPUtime=41.5 cores=4,6
/proc/14276/stat : 14276 (FH) R 14274 14276 14276 0 -1 4202496 668 0 0 0 4150 0 0 0 25 0 1 0 886610537 253169664 406 33554432000 134512640 134936988 4287850304 18446744073709551615 134554623 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/14276/statm: 61809 406 41 104 0 61313 0
[pid=13969/tid=13976] ppid=13967 vsize=351756 CPUtime=0 cores=4,6
/proc/13969/task/13976/stat : 13976 (python) S 13967 13969 12804 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 8 0 886550551 360198144 7464 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
[pid=13969/tid=13990] ppid=13967 vsize=351756 CPUtime=0 cores=4,6
/proc/13969/task/13990/stat : 13990 (python) S 13967 13969 12804 0 -1 4202560 28 931 0 0 0 0 0 0 20 0 8 0 886560565 360198144 7464 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
[pid=13969/tid=14046] ppid=13967 vsize=351756 CPUtime=0 cores=4,6
/proc/13969/task/14046/stat : 14046 (python) S 13967 13969 12804 0 -1 4202560 29 931 0 0 0 0 0 0 20 0 8 0 886570579 360198144 7464 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
[pid=13969/tid=14196] ppid=13967 vsize=351756 CPUtime=0 cores=4,6
/proc/13969/task/14196/stat : 14196 (python) S 13967 13969 12804 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 8 0 886580592 360198144 7464 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
[pid=13969/tid=14256] ppid=13967 vsize=351756 CPUtime=0 cores=4,6
/proc/13969/task/14256/stat : 14256 (python) S 13967 13969 12804 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 8 0 886590606 360198144 7464 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
[pid=13969/tid=14266] ppid=13967 vsize=351756 CPUtime=0 cores=4,6
/proc/13969/task/14266/stat : 14266 (python) S 13967 13969 12804 0 -1 4202560 24 931 0 0 0 0 0 0 20 0 8 0 886600621 360198144 7464 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
[pid=13969/tid=14275] ppid=13967 vsize=351756 CPUtime=0 cores=4,6
/proc/13969/task/14275/stat : 14275 (python) S 13967 13969 12804 0 -1 4202560 30 931 0 0 0 0 0 0 20 0 8 0 886610536 360198144 7464 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 642.61
Current children cumulated vsize (KiB) 7089864



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+700.601 s]
/proc/loadavg: 3.39 3.95 3.95 4/231 14307
/proc/meminfo: memFree=31326588/32951132 swapFree=67052544/67111528
[pid=13969] ppid=13967 vsize=362012 CPUtime=0.84 cores=4,6
/proc/13969/stat : 13969 (python) S 13967 13969 12804 0 -1 4202496 16184 931 1 0 76 8 0 0 20 0 9 0 886550457 370700288 7470 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/13969/statm: 90503 7470 1892 364 0 26400 0
[pid=13975] ppid=13969 vsize=290220 CPUtime=0.12 cores=4,6
/proc/13975/stat : 13975 (python) S 13969 13969 12804 0 -1 4202560 1139 0 0 0 7 5 0 0 19 0 2 0 886550550 297185280 5966 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/13975/statm: 72555 5966 415 364 0 8473 0
[pid=13977] ppid=13975 vsize=1018084 CPUtime=100.12 cores=4,6
/proc/13977/stat : 13977 (gnovelty+2) T 13975 13977 13977 0 -1 4202496 551 0 0 0 10012 0 0 0 25 0 1 0 886550551 1042518016 287 33554432000 134512640 135663704 4286850304 18446744073709551615 134525258 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/13977/statm: 254521 287 113 282 0 254237 0
[pid=13975/tid=13988] ppid=13969 vsize=290220 CPUtime=0 cores=4,6
/proc/13975/task/13988/stat : 13988 (python) S 13969 13969 12804 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 886560564 297185280 5966 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 514 18446604453784002880 0 0 -1 6 0 0 0
[pid=13989] ppid=13969 vsize=290232 CPUtime=0.12 cores=4,6
/proc/13989/stat : 13989 (python) S 13969 13969 12804 0 -1 4202560 1073 0 0 0 7 5 0 0 19 0 2 0 886560564 297197568 5951 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/13989/statm: 72558 5951 400 364 0 8473 0
[pid=13991] ppid=13989 vsize=1018084 CPUtime=100 cores=4,6
/proc/13991/stat : 13991 (gnovelty+2) T 13989 13991 13991 0 -1 4202496 552 0 0 0 10000 0 0 0 25 0 1 0 886560565 1042518016 287 33554432000 134512640 135663704 4293521472 18446744073709551615 134525493 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/13991/statm: 254521 287 113 282 0 254237 0
[pid=13989/tid=14044] ppid=13969 vsize=290232 CPUtime=0 cores=4,6
/proc/13989/task/14044/stat : 14044 (python) S 13969 13969 12804 0 -1 4202560 51 0 0 0 0 0 0 0 19 0 2 0 886570578 297197568 5951 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 514 18446604453890837568 0 0 -1 6 0 0 0
[pid=14045] ppid=13969 vsize=300488 CPUtime=0.13 cores=4,6
/proc/14045/stat : 14045 (python) S 13969 13969 12804 0 -1 4202560 1085 0 0 0 8 5 0 0 19 0 2 0 886570579 307699712 5955 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/14045/statm: 75122 5955 400 364 0 11034 0
[pid=14047] ppid=14045 vsize=417056 CPUtime=99.98 cores=4,6
/proc/14047/stat : 14047 (adaptg2wsat2009) T 14045 14047 14047 0 -1 4202496 719 0 0 0 9998 0 0 0 25 0 1 0 886570579 427065344 450 33554432000 134512640 135107411 4291031888 18446744073709551615 134531741 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/14047/statm: 104264 450 59 146 0 103140 0
[pid=14045/tid=14194] ppid=13969 vsize=300488 CPUtime=0 cores=4,6
/proc/14045/task/14194/stat : 14194 (python) S 13969 13969 12804 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 886580591 307699712 5955 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 514 18446604453781078208 0 0 -1 6 0 0 0
[pid=14195] ppid=13969 vsize=310744 CPUtime=0.13 cores=4,6
/proc/14195/stat : 14195 (python) S 13969 13969 12804 0 -1 4202560 1097 0 0 0 6 7 0 0 19 0 2 0 886580592 318201856 5959 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/14195/statm: 77686 5959 400 364 0 13595 0
[pid=14197] ppid=14195 vsize=417056 CPUtime=100.11 cores=4,6
/proc/14197/stat : 14197 (adaptg2wsat2009) T 14195 14197 14197 0 -1 4202496 718 0 0 0 10009 2 0 0 25 0 1 0 886580592 427065344 452 33554432000 134512640 135107411 4292083936 18446744073709551615 134531604 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/14197/statm: 104264 452 59 146 0 103140 0
[pid=14195/tid=14254] ppid=13969 vsize=310744 CPUtime=0 cores=4,6
/proc/14195/task/14254/stat : 14254 (python) S 13969 13969 12804 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 886590605 318201856 5959 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 514 18446604453851017536 0 0 -1 6 0 0 0
[pid=14255] ppid=13969 vsize=321000 CPUtime=0.13 cores=4,6
/proc/14255/stat : 14255 (python) S 13969 13969 12804 0 -1 4202560 1088 0 0 0 9 4 0 0 19 0 2 0 886590606 328704000 5963 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/14255/statm: 80250 5963 400 364 0 16156 0
[pid=14257] ppid=14255 vsize=417056 CPUtime=100.12 cores=4,6
/proc/14257/stat : 14257 (adaptg2wsat2009) T 14255 14257 14257 0 -1 4202496 719 0 0 0 10012 0 0 0 25 0 1 0 886590607 427065344 450 33554432000 134512640 135107411 4291250528 18446744073709551615 134532349 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/14257/statm: 104264 450 59 146 0 103140 0
[pid=14255/tid=14264] ppid=13969 vsize=321000 CPUtime=0 cores=4,6
/proc/14255/task/14264/stat : 14264 (python) S 13969 13969 12804 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 886600620 328704000 5963 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 514 18446604443456656704 0 0 -1 6 0 0 0
[pid=14265] ppid=13969 vsize=331256 CPUtime=0.13 cores=4,6
/proc/14265/stat : 14265 (python) S 13969 13969 12804 0 -1 4202560 1084 0 0 0 9 4 0 0 19 0 2 0 886600620 339206144 5967 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/14265/statm: 82814 5967 400 364 0 18717 0
[pid=14267] ppid=14265 vsize=1018084 CPUtime=99.14 cores=4,6
/proc/14267/stat : 14267 (gnovelty+2) T 14265 14267 14267 0 -1 4202496 549 0 0 0 9914 0 0 0 25 0 1 0 886600621 1042518016 287 33554432000 134512640 135663704 4291053248 18446744073709551615 134525684 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/14267/statm: 254521 287 113 282 0 254237 0
[pid=14265/tid=14273] ppid=13969 vsize=331256 CPUtime=0 cores=4,6
/proc/14265/task/14273/stat : 14273 (python) S 13969 13969 12804 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 886610535 339206144 5967 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 514 18446604452741558464 0 0 -1 6 0 0 0
[pid=14274] ppid=13969 vsize=341512 CPUtime=0.13 cores=4,6
/proc/14274/stat : 14274 (python) S 13969 13969 12804 0 -1 4202560 1078 0 0 0 8 5 0 0 19 0 2 0 886610536 349708288 5971 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/14274/statm: 85378 5971 400 364 0 21278 0
[pid=14276] ppid=14274 vsize=247236 CPUtime=99.13 cores=4,6
/proc/14276/stat : 14276 (FH) T 14274 14276 14276 0 -1 4202496 668 0 0 0 9913 0 0 0 25 0 1 0 886610537 253169664 406 33554432000 134512640 134936988 4287850304 18446744073709551615 134545189 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/14276/statm: 61809 406 41 104 0 61313 0
[pid=14274/tid=14304] ppid=13969 vsize=341512 CPUtime=0 cores=4,6
/proc/14274/task/14304/stat : 14304 (python) S 13969 13969 12804 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 886620450 349708288 5971 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 514 18446604453715992896 0 0 -1 6 0 0 0
[pid=14305] ppid=13969 vsize=351768 CPUtime=0 cores=4,6
/proc/14305/stat : 14305 (python) S 13969 13969 12804 0 -1 4202560 830 0 0 0 0 0 0 0 19 0 1 0 886620451 360210432 5924 33554432000 4194304 5683644 140737359489440 18446744073709551615 242970570310 0 0 16781312 514 10559481253915000831 0 0 17 4 0 0 0
/proc/14305/statm: 87942 5924 350 364 0 23839 0
[pid=14307] ppid=14305 vsize=416916 CPUtime=0.65 cores=4,6
/proc/14307/stat : 14307 (adaptg2wsat2009) R 14305 14307 14307 0 -1 4202496 643 0 0 0 65 0 0 0 22 0 1 0 886620452 426921984 379 33554432000 134512640 135107411 4290811792 18446744073709551615 134523455 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/14307/statm: 104229 379 49 146 0 103105 0
[pid=13969/tid=13976] ppid=13967 vsize=362012 CPUtime=0 cores=4,6
/proc/13969/task/13976/stat : 13976 (python) S 13967 13969 12804 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 9 0 886550551 370700288 7470 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
[pid=13969/tid=13990] ppid=13967 vsize=362012 CPUtime=0 cores=4,6
/proc/13969/task/13990/stat : 13990 (python) S 13967 13969 12804 0 -1 4202560 28 931 0 0 0 0 0 0 20 0 9 0 886560565 370700288 7470 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
[pid=13969/tid=14046] ppid=13967 vsize=362012 CPUtime=0 cores=4,6
/proc/13969/task/14046/stat : 14046 (python) S 13967 13969 12804 0 -1 4202560 29 931 0 0 0 0 0 0 20 0 9 0 886570579 370700288 7470 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
[pid=13969/tid=14196] ppid=13967 vsize=362012 CPUtime=0 cores=4,6
/proc/13969/task/14196/stat : 14196 (python) S 13967 13969 12804 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 9 0 886580592 370700288 7470 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
[pid=13969/tid=14256] ppid=13967 vsize=362012 CPUtime=0 cores=4,6
/proc/13969/task/14256/stat : 14256 (python) S 13967 13969 12804 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 9 0 886590606 370700288 7470 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
[pid=13969/tid=14266] ppid=13967 vsize=362012 CPUtime=0 cores=4,6
/proc/13969/task/14266/stat : 14266 (python) S 13967 13969 12804 0 -1 4202560 24 931 0 0 0 0 0 0 20 0 9 0 886600621 370700288 7470 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
[pid=13969/tid=14275] ppid=13967 vsize=362012 CPUtime=0 cores=4,6
/proc/13969/task/14275/stat : 14275 (python) S 13967 13969 12804 0 -1 4202560 30 931 0 0 0 0 0 0 20 0 9 0 886610536 370700288 7470 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
[pid=13969/tid=14306] ppid=13967 vsize=362012 CPUtime=0 cores=4,6
/proc/13969/task/14306/stat : 14306 (python) S 13967 13969 12804 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 9 0 886620451 370700288 7470 33554432000 4194304 5683644 140737359489440 18446744073709551615 242982373809 0 0 16781312 2 18446604452741559360 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 700.98
Current children cumulated vsize (KiB) 7868804

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 13969 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=765883
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=87986
# CPU time returned by wait4() is 0.853869
# while last known CPU time is 700.98
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 700.662
CPU time (s): 700.98
CPU user time (s): 700.53
CPU system time (s): 0.45
CPU usage (%): 100.045
Max. virtual memory (cumulated for all children) (KiB): 7868804

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.765883
system time used= 0.087986
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 17121
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1002
involuntary context switches= 32

runsolver used 1.41378 second user time and 2.76758 second system time

The end

Launcher Data 

Begin job on node128 at 2011-06-11 01:45:18
IDJOB=3484404
IDBENCH=82915
IDSOLVER=1975
FILE ID=node128/3484404-1307749518
RUNJOBID= node128-1307747008-12823
PBS_JOBID= 13550829
Free space on /tmp= 74032 MiB

SOLVER NAME= borg-sat 11.03.02-seq-random (fixed, 1200 s timeout)
BENCH NAME= SAT11/random/medium/unif-k7-r89-v80-c7120-S1637697223-097.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3484404-1307749518/watcher-3484404-1307749518 -o /tmp/evaluation-result-3484404-1307749518/solver-3484404-1307749518 -C 1200 -W 1300 -M 7680  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3484404-1307749518.cnf -seed 770318917 -budget 1200

TIME LIMIT= 1200 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= a7e7ad4822818289e739fae39e69d3b8
RANDOM SEED=770318917

node128.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5434.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      31389472 kB
Buffers:        244808 kB
Cached:         937576 kB
SwapCached:      11368 kB
Active:         509012 kB
Inactive:       841968 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      31389472 kB
SwapTotal:    67111528 kB
SwapFree:     67052544 kB
Dirty:           65416 kB
Writeback:           0 kB
AnonPages:      165924 kB
Mapped:          18080 kB
Slab:           140340 kB
PageTables:       9912 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS:  4704756 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74032 MiB
End job on node128 at 2011-06-11 01:57:01