Trace number 3484095

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-random (fixed, 1200 s timeout)? (MO) 702.77 702.865

General information on the benchmark

Namerandom/large/
unif-k7-r85-v300-c25500-S376724493-075-UNKNOWN.cnf
MD5SUMb318fd1a86aee981aadb17224d952a44
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables300
Number of clauses25500
Sum of the clauses size178500
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 525500

Solver Data 

0.29/0.46	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.29/0.46	c INFO: solving HOME/instance-3484095-1307747278.cnf
0.29/0.46	c INFO: computing task features
0.47/0.64	c INFO: computed variable-clause graph statistics
0.47/0.64	c INFO: computed clause balance statistics
1.37/1.53	c INFO: computed variable balance statistics
1.67/1.88	c INFO: computed small-clause counts
1.67/1.89	c INFO: computed Horn-clause counts
1.77/1.92	c INFO: computed variable graph statistics
1.77/1.92	c INFO: collected features for HOME/instance-3484095-1307747278.cnf in 1.45 s
1.77/1.92	c INFO: running gnovelty+2@0 for 97 with 1198 remaining (b = 0.37)
101.87/102.10	c INFO: running gnovelty+2@0 for 97 with 1098 remaining (b = 0.13)
201.22/201.29	c INFO: running adaptg2wsat2009++@0 for 97 with 999 remaining (b = 0.08)
301.44/301.44	c INFO: running adaptg2wsat2009++@0 for 97 with 899 remaining (b = 0.07)
401.52/401.59	c INFO: running adaptg2wsat2009++@0 for 97 with 799 remaining (b = 0.05)
501.70/501.72	c INFO: running gnovelty+2@0 for 97 with 699 remaining (b = 0.04)
601.77/601.89	c INFO: running FH@0 for 97 with 599 remaining (b = 0.03)
701.95/702.04	c INFO: running adaptg2wsat2009++@0 for 97 with 499 remaining (b = 0.03)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 988)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3484095-1307747278/watcher-3484095-1307747278 -o /tmp/evaluation-result-3484095-1307747278/solver-3484095-1307747278 -C 1200 -W 1300 -M 7680 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3484095-1307747278.cnf -seed 1354758474 -budget 1200 

running on 2 cores: 4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1200 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1230 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.98 2.49 1.07 4/218 18106
/proc/meminfo: memFree=31296944/32951124 swapFree=67055520/67111528
[pid=18106] ppid=18104 vsize=15380 CPUtime=0 cores=4,6
/proc/18106/stat : 18106 (python) R 18104 18106 17831 0 -1 4202496 820 0 0 0 0 0 0 0 19 0 1 0 867143478 15749120 704 33554432000 4194304 5683644 140734758856576 18446744073709551615 270338427015 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/18106/statm: 3845 704 334 364 0 332 0

[startup+0.0585209 s]
/proc/loadavg: 3.98 2.49 1.07 4/218 18106
/proc/meminfo: memFree=31296944/32951124 swapFree=67055520/67111528
[pid=18106] ppid=18104 vsize=36836 CPUtime=0.04 cores=4,6
/proc/18106/stat : 18106 (python) S 18104 18106 17831 0 -1 4202496 1623 0 0 0 4 0 0 0 19 0 1 0 867143478 37720064 1452 33554432000 4194304 5683644 140734758856576 18446744073709551615 270333460103 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/18106/statm: 9209 1452 452 364 0 1026 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 36836

[startup+0.100515 s]
/proc/loadavg: 3.98 2.49 1.07 4/218 18106
/proc/meminfo: memFree=31296944/32951124 swapFree=67055520/67111528
[pid=18106] ppid=18104 vsize=104232 CPUtime=0.06 cores=4,6
/proc/18106/stat : 18106 (python) R 18104 18106 17831 0 -1 4202496 3088 0 0 0 5 1 0 0 19 0 1 0 867143478 106733568 2768 33554432000 4194304 5683644 140734758856576 18446744073709551615 5276704 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/18106/statm: 26058 2768 847 364 0 1806 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 104232

[startup+0.300462 s]
/proc/loadavg: 3.98 2.49 1.07 4/218 18106
/proc/meminfo: memFree=31296944/32951124 swapFree=67055520/67111528
[pid=18106] ppid=18104 vsize=194548 CPUtime=0.21 cores=4,6
/proc/18106/stat : 18106 (python) R 18104 18106 17831 0 -1 4202496 6928 932 1 0 17 4 0 0 17 0 1 0 867143478 199217152 4975 33554432000 4194304 5683644 140734758856576 18446744073709551615 270338381824 0 2147483391 16781312 2 0 0 0 17 4 0 0 0
/proc/18106/statm: 48637 4975 1431 364 0 3756 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 194548

[startup+0.70037 s]
/proc/loadavg: 3.98 2.49 1.07 4/218 18106
/proc/meminfo: memFree=31296944/32951124 swapFree=67055520/67111528
[pid=18106] ppid=18104 vsize=285632 CPUtime=0.57 cores=4,6
/proc/18106/stat : 18106 (python) R 18104 18106 17831 0 -1 4202496 12506 932 1 0 51 6 0 0 17 0 1 0 867143478 292487168 8793 33554432000 4194304 5683644 140734758856576 18446744073709551615 47241563490588 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/18106/statm: 71408 8793 1826 364 0 7332 0
Current children cumulated CPU time (s) 0.57
Current children cumulated vsize (KiB) 285632

[startup+1.50023 s]
/proc/loadavg: 3.98 2.49 1.07 5/219 18111
/proc/meminfo: memFree=31268312/32951124 swapFree=67055520/67111528
[pid=18106] ppid=18104 vsize=285632 CPUtime=1.37 cores=4,6
/proc/18106/stat : 18106 (python) R 18104 18106 17831 0 -1 4202496 12506 932 1 0 131 6 0 0 19 0 1 0 867143478 292487168 8793 33554432000 4194304 5683644 140734758856576 18446744073709551615 47241560866850 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/18106/statm: 71408 8793 1826 364 0 7332 0
Current children cumulated CPU time (s) 1.37
Current children cumulated vsize (KiB) 285632

[startup+3.10092 s]
/proc/loadavg: 3.98 2.49 1.07 5/222 18114
/proc/meminfo: memFree=31269284/32951124 swapFree=67055520/67111528
[pid=18106] ppid=18104 vsize=290516 CPUtime=1.79 cores=4,6
/proc/18106/stat : 18106 (python) S 18104 18106 17831 0 -1 4202496 12832 932 1 0 173 6 0 0 21 0 2 0 867143478 297488384 7494 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/18106/statm: 72629 7494 1869 364 0 8547 0
[pid=18112] ppid=18106 vsize=280756 CPUtime=0 cores=4,6
/proc/18112/stat : 18112 (python) S 18106 18106 17831 0 -1 4202560 721 0 0 0 0 0 0 0 18 0 1 0 867143671 287494144 5977 33554432000 4194304 5683644 140734758856576 18446744073709551615 270338403855 0 0 16781312 514 1078409804910657794 0 0 17 4 0 0 0
/proc/18112/statm: 70189 5977 355 364 0 6107 0
[pid=18114] ppid=18112 vsize=1019708 CPUtime=1.16 cores=4,6
/proc/18114/stat : 18114 (gnovelty+2) R 18112 18114 18114 0 -1 4202496 953 0 0 0 116 0 0 0 25 0 1 0 867143671 1044180992 690 33554432000 134512640 135663704 4294640960 18446744073709551615 134525258 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/18114/statm: 254927 690 113 282 0 254643 0
[pid=18106/tid=18113] ppid=18104 vsize=290516 CPUtime=0 cores=4,6
/proc/18106/task/18113/stat : 18113 (python) S 18104 18106 17831 0 -1 4202560 26 932 0 0 0 0 0 0 21 0 2 0 867143671 297488384 7494 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2.95
Current children cumulated vsize (KiB) 1590980

[startup+6.30022 s]
/proc/loadavg: 3.98 2.51 1.08 5/222 18114
/proc/meminfo: memFree=31268652/32951124 swapFree=67055520/67111528
[pid=18106] ppid=18104 vsize=290516 CPUtime=1.79 cores=4,6
/proc/18106/stat : 18106 (python) S 18104 18106 17831 0 -1 4202496 12832 932 1 0 173 6 0 0 21 0 2 0 867143478 297488384 7494 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/18106/statm: 72629 7494 1869 364 0 8547 0
[pid=18112] ppid=18106 vsize=280756 CPUtime=0 cores=4,6
/proc/18112/stat : 18112 (python) S 18106 18106 17831 0 -1 4202560 860 0 0 0 0 0 0 0 18 0 1 0 867143671 287494144 6000 33554432000 4194304 5683644 140734758856576 18446744073709551615 270338403855 0 0 16781312 514 1078409804910690303 0 0 17 4 0 0 0
/proc/18112/statm: 70189 6000 378 364 0 6107 0
[pid=18114] ppid=18112 vsize=1019708 CPUtime=4.36 cores=4,6
/proc/18114/stat : 18114 (gnovelty+2) R 18112 18114 18114 0 -1 4202496 953 0 0 0 436 0 0 0 25 0 1 0 867143671 1044180992 690 33554432000 134512640 135663704 4294640960 18446744073709551615 134525218 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/18114/statm: 254927 690 113 282 0 254643 0
[pid=18106/tid=18113] ppid=18104 vsize=290516 CPUtime=0 cores=4,6
/proc/18106/task/18113/stat : 18113 (python) S 18104 18106 17831 0 -1 4202560 26 932 0 0 0 0 0 0 21 0 2 0 867143671 297488384 7494 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 6.15
Current children cumulated vsize (KiB) 1590980

[startup+12.701 s]
/proc/loadavg: 3.98 2.54 1.10 5/222 18114
/proc/meminfo: memFree=31268148/32951124 swapFree=67055520/67111528
[pid=18106] ppid=18104 vsize=290516 CPUtime=1.79 cores=4,6
/proc/18106/stat : 18106 (python) S 18104 18106 17831 0 -1 4202496 12832 932 1 0 173 6 0 0 21 0 2 0 867143478 297488384 7494 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/18106/statm: 72629 7494 1869 364 0 8547 0
[pid=18112] ppid=18106 vsize=280756 CPUtime=0.01 cores=4,6
/proc/18112/stat : 18112 (python) S 18106 18106 17831 0 -1 4202560 864 0 0 0 1 0 0 0 18 0 1 0 867143671 287494144 6000 33554432000 4194304 5683644 140734758856576 18446744073709551615 270338403855 0 0 16781312 514 5000948319032148226 0 0 17 4 0 0 0
/proc/18112/statm: 70189 6000 378 364 0 6107 0
[pid=18114] ppid=18112 vsize=1019708 CPUtime=10.75 cores=4,6
/proc/18114/stat : 18114 (gnovelty+2) R 18112 18114 18114 0 -1 4202496 953 0 0 0 1075 0 0 0 25 0 1 0 867143671 1044180992 690 33554432000 134512640 135663704 4294640960 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/18114/statm: 254927 690 113 282 0 254643 0
[pid=18106/tid=18113] ppid=18104 vsize=290516 CPUtime=0 cores=4,6
/proc/18106/task/18113/stat : 18113 (python) S 18104 18106 17831 0 -1 4202560 26 932 0 0 0 0 0 0 21 0 2 0 867143671 297488384 7494 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 12.55
Current children cumulated vsize (KiB) 1590980

[startup+25.5005 s]
/proc/loadavg: 3.98 2.61 1.14 5/230 18122
/proc/meminfo: memFree=31244888/32951124 swapFree=67055520/67111528
[pid=18106] ppid=18104 vsize=290516 CPUtime=1.79 cores=4,6
/proc/18106/stat : 18106 (python) S 18104 18106 17831 0 -1 4202496 12832 932 1 0 173 6 0 0 21 0 2 0 867143478 297488384 7494 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/18106/statm: 72629 7494 1869 364 0 8547 0
[pid=18112] ppid=18106 vsize=280756 CPUtime=0.03 cores=4,6
/proc/18112/stat : 18112 (python) S 18106 18106 17831 0 -1 4202560 872 0 0 0 2 1 0 0 18 0 1 0 867143671 287494144 6000 33554432000 4194304 5683644 140734758856576 18446744073709551615 270338403855 0 0 16781312 514 5000948319032148226 0 0 17 4 0 0 0
/proc/18112/statm: 70189 6000 378 364 0 6107 0
[pid=18114] ppid=18112 vsize=1019708 CPUtime=23.54 cores=4,6
/proc/18114/stat : 18114 (gnovelty+2) R 18112 18114 18114 0 -1 4202496 953 0 0 0 2354 0 0 0 25 0 1 0 867143671 1044180992 690 33554432000 134512640 135663704 4294640960 18446744073709551615 134547181 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/18114/statm: 254927 690 113 282 0 254643 0
[pid=18106/tid=18113] ppid=18104 vsize=290516 CPUtime=0 cores=4,6
/proc/18106/task/18113/stat : 18113 (python) S 18104 18106 17831 0 -1 4202560 26 932 0 0 0 0 0 0 21 0 2 0 867143671 297488384 7494 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 25.36
Current children cumulated vsize (KiB) 1590980

[startup+51.1004 s]
/proc/loadavg: 3.99 2.72 1.22 5/230 18123
/proc/meminfo: memFree=31244416/32951124 swapFree=67055520/67111528
[pid=18106] ppid=18104 vsize=290516 CPUtime=1.79 cores=4,6
/proc/18106/stat : 18106 (python) S 18104 18106 17831 0 -1 4202496 12832 932 1 0 173 6 0 0 21 0 2 0 867143478 297488384 7494 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/18106/statm: 72629 7494 1869 364 0 8547 0
[pid=18112] ppid=18106 vsize=280756 CPUtime=0.05 cores=4,6
/proc/18112/stat : 18112 (python) S 18106 18106 17831 0 -1 4202560 885 0 0 0 3 2 0 0 18 0 1 0 867143671 287494144 6000 33554432000 4194304 5683644 140734758856576 18446744073709551615 270338403855 0 0 16781312 514 5000948319032148226 0 0 17 4 0 0 0
/proc/18112/statm: 70189 6000 378 364 0 6107 0
[pid=18114] ppid=18112 vsize=1019708 CPUtime=49.11 cores=4,6
/proc/18114/stat : 18114 (gnovelty+2) R 18112 18114 18114 0 -1 4202496 953 0 0 0 4911 0 0 0 25 0 1 0 867143671 1044180992 690 33554432000 134512640 135663704 4294640960 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/18114/statm: 254927 690 113 282 0 254643 0
[pid=18106/tid=18113] ppid=18104 vsize=290516 CPUtime=0 cores=4,6
/proc/18106/task/18113/stat : 18113 (python) S 18104 18106 17831 0 -1 4202560 26 932 0 0 0 0 0 0 21 0 2 0 867143671 297488384 7494 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.95
Current children cumulated vsize (KiB) 1590980

[startup+102.308 s]
/proc/loadavg: 3.99 2.91 1.36 5/238 18132
/proc/meminfo: memFree=31204204/32951124 swapFree=67055520/67111528
[pid=18106] ppid=18104 vsize=300772 CPUtime=1.8 cores=4,6
/proc/18106/stat : 18106 (python) S 18104 18106 17831 0 -1 4202496 13723 932 1 0 174 6 0 0 20 0 3 0 867143478 307990528 7509 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/18106/statm: 75193 7509 1880 364 0 11108 0
[pid=18112] ppid=18106 vsize=291000 CPUtime=0.13 cores=4,6
/proc/18112/stat : 18112 (python) S 18106 18106 17831 0 -1 4202560 1149 0 0 0 7 6 0 0 19 0 2 0 867143671 297984000 6042 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18112/statm: 72750 6042 415 364 0 8668 0
[pid=18114] ppid=18112 vsize=1019708 CPUtime=100.05 cores=4,6
/proc/18114/stat : 18114 (gnovelty+2) T 18112 18114 18114 0 -1 4202496 953 0 0 0 10005 0 0 0 25 0 1 0 867143671 1044180992 690 33554432000 134512640 135663704 4294640960 18446744073709551615 134524947 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/18114/statm: 254927 690 113 282 0 254643 0
[pid=18112/tid=18129] ppid=18106 vsize=291000 CPUtime=0 cores=4,6
/proc/18112/task/18129/stat : 18129 (python) S 18106 18106 17831 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 867153688 297984000 6042 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 514 18446604440278878208 0 0 -1 6 0 0 0
[pid=18130] ppid=18106 vsize=291088 CPUtime=0 cores=4,6
/proc/18130/stat : 18130 (python) S 18106 18106 17831 0 -1 4202560 739 0 0 0 0 0 0 0 18 0 1 0 867153688 298074112 5975 33554432000 4194304 5683644 140734758856576 18446744073709551615 270338403910 0 0 16781312 514 4261870076153561087 0 0 17 6 0 0 0
/proc/18130/statm: 72772 5975 349 364 0 8687 0
[pid=18132] ppid=18130 vsize=1019708 CPUtime=0.2 cores=4,6
/proc/18132/stat : 18132 (gnovelty+2) R 18130 18132 18132 0 -1 4202496 955 0 0 0 20 0 0 0 20 0 1 0 867153689 1044180992 689 33554432000 134512640 135663704 4294716352 18446744073709551615 134545786 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/18132/statm: 254927 689 113 282 0 254643 0
[pid=18106/tid=18113] ppid=18104 vsize=300772 CPUtime=0 cores=4,6
/proc/18106/task/18113/stat : 18113 (python) S 18104 18106 17831 0 -1 4202560 26 932 0 0 0 0 0 0 21 0 3 0 867143671 307990528 7509 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 6 0 0 0
[pid=18106/tid=18131] ppid=18104 vsize=300772 CPUtime=0 cores=4,6
/proc/18106/task/18131/stat : 18131 (python) S 18104 18106 17831 0 -1 4202560 30 932 0 0 0 0 0 0 20 0 3 0 867153689 307990528 7509 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 102.18
Current children cumulated vsize (KiB) 2922276

[startup+162.301 s]
/proc/loadavg: 3.99 3.11 1.52 5/246 18142
/proc/meminfo: memFree=31178404/32951124 swapFree=67055520/67111528
[pid=18106] ppid=18104 vsize=300772 CPUtime=1.8 cores=4,6
/proc/18106/stat : 18106 (python) S 18104 18106 17831 0 -1 4202496 13723 932 1 0 174 6 0 0 20 0 3 0 867143478 307990528 7509 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/18106/statm: 75193 7509 1880 364 0 11108 0
[pid=18112] ppid=18106 vsize=291000 CPUtime=0.13 cores=4,6
/proc/18112/stat : 18112 (python) S 18106 18106 17831 0 -1 4202560 1149 0 0 0 7 6 0 0 19 0 2 0 867143671 297984000 6042 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18112/statm: 72750 6042 415 364 0 8668 0
[pid=18114] ppid=18112 vsize=1019708 CPUtime=100.05 cores=4,6
/proc/18114/stat : 18114 (gnovelty+2) T 18112 18114 18114 0 -1 4202496 953 0 0 0 10005 0 0 0 25 0 1 0 867143671 1044180992 690 33554432000 134512640 135663704 4294640960 18446744073709551615 134524947 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/18114/statm: 254927 690 113 282 0 254643 0
[pid=18112/tid=18129] ppid=18106 vsize=291000 CPUtime=0 cores=4,6
/proc/18112/task/18129/stat : 18129 (python) S 18106 18106 17831 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 867153688 297984000 6042 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 514 18446604440278878208 0 0 -1 6 0 0 0
[pid=18130] ppid=18106 vsize=291088 CPUtime=0.08 cores=4,6

################
# More data... #
################

/proc/18130/statm: 72772 6027 400 364 0 8687 0
[pid=18132] ppid=18130 vsize=1019708 CPUtime=99.18 cores=4,6
/proc/18132/stat : 18132 (gnovelty+2) T 18130 18132 18132 0 -1 4202496 955 0 0 0 9918 0 0 0 25 0 1 0 867153689 1044180992 689 33554432000 134512640 135663704 4294716352 18446744073709551615 134524882 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/18132/statm: 254927 689 113 282 0 254643 0
[pid=18130/tid=18145] ppid=18106 vsize=291088 CPUtime=0 cores=4,6
/proc/18130/task/18145/stat : 18145 (python) S 18106 18106 17831 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 867163607 298074112 6027 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 514 18446604453904987200 0 0 -1 6 0 0 0
[pid=18146] ppid=18106 vsize=301428 CPUtime=0.14 cores=4,6
/proc/18146/stat : 18146 (python) S 18106 18106 17831 0 -1 4202560 1076 0 0 0 9 5 0 0 19 0 2 0 867163608 308662272 6050 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18146/statm: 75357 6050 400 364 0 11269 0
[pid=18148] ppid=18146 vsize=420220 CPUtime=100 cores=4,6
/proc/18148/stat : 18148 (adaptg2wsat2009) T 18146 18148 18148 0 -1 4202496 1659 0 0 0 10000 0 0 0 25 0 1 0 867163608 430305280 1398 33554432000 134512640 135107411 4287937360 18446744073709551615 134532302 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/18148/statm: 105055 1398 59 146 0 103931 0
[pid=18146/tid=18160] ppid=18106 vsize=301428 CPUtime=0 cores=4,6
/proc/18146/task/18160/stat : 18160 (python) S 18106 18106 17831 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 867173622 308662272 6050 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 514 18446604453923466240 0 0 -1 6 0 0 0
[pid=18161] ppid=18106 vsize=311616 CPUtime=0.14 cores=4,6
/proc/18161/stat : 18161 (python) S 18106 18106 17831 0 -1 4202560 1093 0 0 0 8 6 0 0 19 0 2 0 867173623 319094784 6075 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18161/statm: 77904 6075 400 364 0 13813 0
[pid=18163] ppid=18161 vsize=420220 CPUtime=99.98 cores=4,6
/proc/18163/stat : 18163 (adaptg2wsat2009) T 18161 18163 18163 0 -1 4202496 1666 0 0 0 9998 0 0 0 25 0 1 0 867173623 430305280 1399 33554432000 134512640 135107411 4292753584 18446744073709551615 134531797 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/18163/statm: 105055 1399 59 146 0 103931 0
[pid=18161/tid=18177] ppid=18106 vsize=311616 CPUtime=0 cores=4,6
/proc/18161/task/18177/stat : 18177 (python) S 18106 18106 17831 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 867183636 319094784 6075 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 514 18446604453933467520 0 0 -1 4 0 0 0
[pid=18178] ppid=18106 vsize=322064 CPUtime=0.13 cores=4,6
/proc/18178/stat : 18178 (python) S 18106 18106 17831 0 -1 4202560 1086 0 0 0 8 5 0 0 19 0 2 0 867183637 329793536 6079 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18178/statm: 80516 6079 400 364 0 16422 0
[pid=18180] ppid=18178 vsize=420220 CPUtime=99.99 cores=4,6
/proc/18180/stat : 18180 (adaptg2wsat2009) T 18178 18180 18180 0 -1 4202496 1666 0 0 0 9999 0 0 0 25 0 1 0 867183638 430305280 1401 33554432000 134512640 135107411 4286910992 18446744073709551615 134531781 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/18180/statm: 105055 1401 59 146 0 103931 0
[pid=18178/tid=18285] ppid=18106 vsize=322064 CPUtime=0 cores=4,6
/proc/18178/task/18285/stat : 18285 (python) S 18106 18106 17831 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 867193650 329793536 6079 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 514 18446604453934152512 0 0 -1 6 0 0 0
[pid=18286] ppid=18106 vsize=332428 CPUtime=0.12 cores=4,6
/proc/18286/stat : 18286 (python) S 18106 18106 17831 0 -1 4202560 1080 0 0 0 7 5 0 0 19 0 2 0 867193651 340406272 6114 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18286/statm: 83107 6114 400 364 0 19010 0
[pid=18288] ppid=18286 vsize=1019708 CPUtime=100.01 cores=4,6
/proc/18288/stat : 18288 (gnovelty+2) T 18286 18288 18288 0 -1 4202496 949 0 0 0 10001 0 0 0 25 0 1 0 867193651 1044180992 689 33554432000 134512640 135663704 4289977904 18446744073709551615 134524868 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/18288/statm: 254927 689 113 282 0 254643 0
[pid=18286/tid=18303] ppid=18106 vsize=332428 CPUtime=0 cores=4,6
/proc/18286/task/18303/stat : 18303 (python) S 18106 18106 17831 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 867203667 340406272 6114 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 514 18446604453965060224 0 0 -1 6 0 0 0
[pid=18304] ppid=18106 vsize=342796 CPUtime=0.15 cores=4,6
/proc/18304/stat : 18304 (python) S 18106 18106 17831 0 -1 4202560 1081 0 0 0 8 7 0 0 19 0 2 0 867203668 351023104 6145 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18304/statm: 85699 6145 400 364 0 21599 0
[pid=18306] ppid=18304 vsize=249868 CPUtime=99.99 cores=4,6
/proc/18306/stat : 18306 (FH) T 18304 18306 18306 0 -1 4202496 1525 0 0 0 9999 0 0 0 25 0 1 0 867203668 255864832 1263 33554432000 134512640 134936988 4291759536 18446744073709551615 134553238 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/18306/statm: 62467 1263 41 104 0 61971 0
[pid=18106/tid=18113] ppid=18104 vsize=362700 CPUtime=0 cores=4,6
/proc/18106/task/18113/stat : 18113 (python) S 18104 18106 17831 0 -1 4202560 26 932 0 0 0 0 0 0 21 0 9 0 867143671 371404800 7671 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 6 0 0 0
[pid=18106/tid=18131] ppid=18104 vsize=362700 CPUtime=0 cores=4,6
/proc/18106/task/18131/stat : 18131 (python) S 18104 18106 17831 0 -1 4202560 30 932 0 0 0 0 0 0 20 0 9 0 867153689 371404800 7671 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 4 0 0 0
[pid=18106/tid=18147] ppid=18104 vsize=362700 CPUtime=0 cores=4,6
/proc/18106/task/18147/stat : 18147 (python) S 18104 18106 17831 0 -1 4202560 29 932 0 0 0 0 0 0 20 0 9 0 867163608 371404800 7671 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 4 0 0 0
[pid=18106/tid=18162] ppid=18104 vsize=362700 CPUtime=0 cores=4,6
/proc/18106/task/18162/stat : 18162 (python) S 18104 18106 17831 0 -1 4202560 26 932 0 0 0 0 0 0 20 0 9 0 867173623 371404800 7671 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 4 0 0 0
[pid=18106/tid=18179] ppid=18104 vsize=362700 CPUtime=0 cores=4,6
/proc/18106/task/18179/stat : 18179 (python) S 18104 18106 17831 0 -1 4202560 22 932 0 0 0 0 0 0 20 0 9 0 867183638 371404800 7671 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 4 0 0 0
[pid=18106/tid=18287] ppid=18104 vsize=362700 CPUtime=0 cores=4,6
/proc/18106/task/18287/stat : 18287 (python) S 18104 18106 17831 0 -1 4202560 25 932 0 0 0 0 0 0 20 0 9 0 867193651 371404800 7671 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 4 0 0 0
[pid=18106/tid=18305] ppid=18104 vsize=362700 CPUtime=0 cores=4,6
/proc/18106/task/18305/stat : 18305 (python) S 18104 18106 17831 0 -1 4202560 30 932 0 0 0 0 0 0 20 0 9 0 867203668 371404800 7671 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 702.01
Current children cumulated vsize (KiB) 7124772



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+702.802 s]
/proc/loadavg: 4.15 3.89 2.62 5/263 18322
/proc/meminfo: memFree=31088428/32951124 swapFree=67055520/67111528
[pid=18106] ppid=18104 vsize=362700 CPUtime=1.86 cores=4,6
/proc/18106/stat : 18106 (python) S 18104 18106 17831 0 -1 4202496 18811 932 1 0 178 8 0 0 20 0 9 0 867143478 371404800 7671 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/18106/statm: 90675 7671 1892 364 0 26572 0
[pid=18112] ppid=18106 vsize=291000 CPUtime=0.13 cores=4,6
/proc/18112/stat : 18112 (python) S 18106 18106 17831 0 -1 4202560 1149 0 0 0 7 6 0 0 19 0 2 0 867143671 297984000 6042 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18112/statm: 72750 6042 415 364 0 8668 0
[pid=18114] ppid=18112 vsize=1019708 CPUtime=100.05 cores=4,6
/proc/18114/stat : 18114 (gnovelty+2) T 18112 18114 18114 0 -1 4202496 953 0 0 0 10005 0 0 0 25 0 1 0 867143671 1044180992 690 33554432000 134512640 135663704 4294640960 18446744073709551615 134524947 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/18114/statm: 254927 690 113 282 0 254643 0
[pid=18112/tid=18129] ppid=18106 vsize=291000 CPUtime=0 cores=4,6
/proc/18112/task/18129/stat : 18129 (python) S 18106 18106 17831 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 867153688 297984000 6042 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 514 18446604440278878208 0 0 -1 6 0 0 0
[pid=18130] ppid=18106 vsize=291088 CPUtime=0.14 cores=4,6
/proc/18130/stat : 18130 (python) S 18106 18106 17831 0 -1 4202560 1094 0 0 0 9 5 0 0 19 0 2 0 867153688 298074112 6027 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/18130/statm: 72772 6027 400 364 0 8687 0
[pid=18132] ppid=18130 vsize=1019708 CPUtime=99.18 cores=4,6
/proc/18132/stat : 18132 (gnovelty+2) T 18130 18132 18132 0 -1 4202496 955 0 0 0 9918 0 0 0 25 0 1 0 867153689 1044180992 689 33554432000 134512640 135663704 4294716352 18446744073709551615 134524882 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/18132/statm: 254927 689 113 282 0 254643 0
[pid=18130/tid=18145] ppid=18106 vsize=291088 CPUtime=0 cores=4,6
/proc/18130/task/18145/stat : 18145 (python) S 18106 18106 17831 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 867163607 298074112 6027 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 514 18446604453904987200 0 0 -1 6 0 0 0
[pid=18146] ppid=18106 vsize=301428 CPUtime=0.14 cores=4,6
/proc/18146/stat : 18146 (python) S 18106 18106 17831 0 -1 4202560 1076 0 0 0 9 5 0 0 19 0 2 0 867163608 308662272 6050 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18146/statm: 75357 6050 400 364 0 11269 0
[pid=18148] ppid=18146 vsize=420220 CPUtime=100 cores=4,6
/proc/18148/stat : 18148 (adaptg2wsat2009) T 18146 18148 18148 0 -1 4202496 1659 0 0 0 10000 0 0 0 25 0 1 0 867163608 430305280 1398 33554432000 134512640 135107411 4287937360 18446744073709551615 134532302 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/18148/statm: 105055 1398 59 146 0 103931 0
[pid=18146/tid=18160] ppid=18106 vsize=301428 CPUtime=0 cores=4,6
/proc/18146/task/18160/stat : 18160 (python) S 18106 18106 17831 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 867173622 308662272 6050 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 514 18446604453923466240 0 0 -1 6 0 0 0
[pid=18161] ppid=18106 vsize=311616 CPUtime=0.14 cores=4,6
/proc/18161/stat : 18161 (python) S 18106 18106 17831 0 -1 4202560 1093 0 0 0 8 6 0 0 19 0 2 0 867173623 319094784 6075 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18161/statm: 77904 6075 400 364 0 13813 0
[pid=18163] ppid=18161 vsize=420220 CPUtime=99.98 cores=4,6
/proc/18163/stat : 18163 (adaptg2wsat2009) T 18161 18163 18163 0 -1 4202496 1666 0 0 0 9998 0 0 0 25 0 1 0 867173623 430305280 1399 33554432000 134512640 135107411 4292753584 18446744073709551615 134531797 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/18163/statm: 105055 1399 59 146 0 103931 0
[pid=18161/tid=18177] ppid=18106 vsize=311616 CPUtime=0 cores=4,6
/proc/18161/task/18177/stat : 18177 (python) S 18106 18106 17831 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 867183636 319094784 6075 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 514 18446604453933467520 0 0 -1 4 0 0 0
[pid=18178] ppid=18106 vsize=322064 CPUtime=0.13 cores=4,6
/proc/18178/stat : 18178 (python) S 18106 18106 17831 0 -1 4202560 1086 0 0 0 8 5 0 0 19 0 2 0 867183637 329793536 6079 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18178/statm: 80516 6079 400 364 0 16422 0
[pid=18180] ppid=18178 vsize=420220 CPUtime=99.99 cores=4,6
/proc/18180/stat : 18180 (adaptg2wsat2009) T 18178 18180 18180 0 -1 4202496 1666 0 0 0 9999 0 0 0 25 0 1 0 867183638 430305280 1401 33554432000 134512640 135107411 4286910992 18446744073709551615 134531781 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/18180/statm: 105055 1401 59 146 0 103931 0
[pid=18178/tid=18285] ppid=18106 vsize=322064 CPUtime=0 cores=4,6
/proc/18178/task/18285/stat : 18285 (python) S 18106 18106 17831 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 867193650 329793536 6079 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 514 18446604453934152512 0 0 -1 6 0 0 0
[pid=18286] ppid=18106 vsize=332428 CPUtime=0.12 cores=4,6
/proc/18286/stat : 18286 (python) S 18106 18106 17831 0 -1 4202560 1080 0 0 0 7 5 0 0 19 0 2 0 867193651 340406272 6114 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18286/statm: 83107 6114 400 364 0 19010 0
[pid=18288] ppid=18286 vsize=1019708 CPUtime=100.01 cores=4,6
/proc/18288/stat : 18288 (gnovelty+2) T 18286 18288 18288 0 -1 4202496 949 0 0 0 10001 0 0 0 25 0 1 0 867193651 1044180992 689 33554432000 134512640 135663704 4289977904 18446744073709551615 134524868 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/18288/statm: 254927 689 113 282 0 254643 0
[pid=18286/tid=18303] ppid=18106 vsize=332428 CPUtime=0 cores=4,6
/proc/18286/task/18303/stat : 18303 (python) S 18106 18106 17831 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 867203667 340406272 6114 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 514 18446604453965060224 0 0 -1 6 0 0 0
[pid=18304] ppid=18106 vsize=342796 CPUtime=0.15 cores=4,6
/proc/18304/stat : 18304 (python) S 18106 18106 17831 0 -1 4202560 1081 0 0 0 8 7 0 0 19 0 2 0 867203668 351023104 6145 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18304/statm: 85699 6145 400 364 0 21599 0
[pid=18306] ppid=18304 vsize=249868 CPUtime=99.99 cores=4,6
/proc/18306/stat : 18306 (FH) T 18304 18306 18306 0 -1 4202496 1525 0 0 0 9999 0 0 0 25 0 1 0 867203668 255864832 1263 33554432000 134512640 134936988 4291759536 18446744073709551615 134553238 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/18306/statm: 62467 1263 41 104 0 61971 0
[pid=18304/tid=18318] ppid=18106 vsize=342796 CPUtime=0 cores=4,6
/proc/18304/task/18318/stat : 18318 (python) S 18106 18106 17831 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 867213682 351023104 6145 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 514 18446604453934149824 0 0 -1 6 0 0 0
[pid=18319] ppid=18106 vsize=353172 CPUtime=0 cores=4,6
/proc/18319/stat : 18319 (python) S 18106 18106 17831 0 -1 4202560 836 0 0 0 0 0 0 0 19 0 1 0 867213683 361648128 6127 33554432000 4194304 5683644 140734758856576 18446744073709551615 270338403910 0 0 16781312 514 4358278554211844095 0 0 17 6 0 0 0
/proc/18319/statm: 88293 6127 351 364 0 24190 0
[pid=18321] ppid=18319 vsize=419820 CPUtime=0.76 cores=4,6
/proc/18321/stat : 18321 (adaptg2wsat2009) R 18319 18321 18321 0 -1 4202496 1436 0 0 0 76 0 0 0 22 0 1 0 867213683 429895680 1172 33554432000 134512640 135107411 4287178352 18446744073709551615 134523448 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/18321/statm: 104955 1172 49 146 0 103831 0
[pid=18106/tid=18113] ppid=18104 vsize=362700 CPUtime=0 cores=4,6
/proc/18106/task/18113/stat : 18113 (python) S 18104 18106 17831 0 -1 4202560 26 932 0 0 0 0 0 0 21 0 9 0 867143671 371404800 7671 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 6 0 0 0
[pid=18106/tid=18131] ppid=18104 vsize=362700 CPUtime=0 cores=4,6
/proc/18106/task/18131/stat : 18131 (python) S 18104 18106 17831 0 -1 4202560 30 932 0 0 0 0 0 0 20 0 9 0 867153689 371404800 7671 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 4 0 0 0
[pid=18106/tid=18147] ppid=18104 vsize=362700 CPUtime=0 cores=4,6
/proc/18106/task/18147/stat : 18147 (python) S 18104 18106 17831 0 -1 4202560 29 932 0 0 0 0 0 0 20 0 9 0 867163608 371404800 7671 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 4 0 0 0
[pid=18106/tid=18162] ppid=18104 vsize=362700 CPUtime=0 cores=4,6
/proc/18106/task/18162/stat : 18162 (python) S 18104 18106 17831 0 -1 4202560 26 932 0 0 0 0 0 0 20 0 9 0 867173623 371404800 7671 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 4 0 0 0
[pid=18106/tid=18179] ppid=18104 vsize=362700 CPUtime=0 cores=4,6
/proc/18106/task/18179/stat : 18179 (python) S 18104 18106 17831 0 -1 4202560 22 932 0 0 0 0 0 0 20 0 9 0 867183638 371404800 7671 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 4 0 0 0
[pid=18106/tid=18287] ppid=18104 vsize=362700 CPUtime=0 cores=4,6
/proc/18106/task/18287/stat : 18287 (python) S 18104 18106 17831 0 -1 4202560 25 932 0 0 0 0 0 0 20 0 9 0 867193651 371404800 7671 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 4 0 0 0
[pid=18106/tid=18305] ppid=18104 vsize=362700 CPUtime=0 cores=4,6
/proc/18106/task/18305/stat : 18305 (python) S 18104 18106 17831 0 -1 4202560 30 932 0 0 0 0 0 0 20 0 9 0 867203668 371404800 7671 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 4 0 0 0
[pid=18106/tid=18320] ppid=18104 vsize=362700 CPUtime=0 cores=4,6
/proc/18106/task/18320/stat : 18320 (python) S 18104 18106 17831 0 -1 4202560 25 932 0 0 0 0 0 0 20 0 9 0 867213683 371404800 7671 33554432000 4194304 5683644 140734758856576 18446744073709551615 270350207409 0 0 16781312 2 18446604453948975872 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 702.77
Current children cumulated vsize (KiB) 7897764

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 18106 and gives
#  childrusage.ru_utime.tv_sec=1
#  childrusage.ru_utime.tv_usec=795727
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=91986
# CPU time returned by wait4() is 1.88771
# while last known CPU time is 702.77
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 702.865
CPU time (s): 702.77
CPU user time (s): 702.3
CPU system time (s): 0.47
CPU usage (%): 99.9865
Max. virtual memory (cumulated for all children) (KiB): 7897764

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1.79573
system time used= 0.091986
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 20413
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1001
involuntary context switches= 43

runsolver used 1.88671 second user time and 3.21151 second system time

The end

Launcher Data 

Begin job on node143 at 2011-06-11 01:07:58
IDJOB=3484095
IDBENCH=82606
IDSOLVER=1975
FILE ID=node143/3484095-1307747278
RUNJOBID= node143-1307746989-17850
PBS_JOBID= 13550817
Free space on /tmp= 74020 MiB

SOLVER NAME= borg-sat 11.03.02-seq-random (fixed, 1200 s timeout)
BENCH NAME= SAT11/random/large/unif-k7-r85-v300-c25500-S376724493-075-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3484095-1307747278/watcher-3484095-1307747278 -o /tmp/evaluation-result-3484095-1307747278/solver-3484095-1307747278 -C 1200 -W 1300 -M 7680  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3484095-1307747278.cnf -seed 1354758474 -budget 1200

TIME LIMIT= 1200 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= b318fd1a86aee981aadb17224d952a44
RANDOM SEED=1354758474

node143.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      31297620 kB
Buffers:        109116 kB
Cached:        1146232 kB
SwapCached:       8636 kB
Active:         597836 kB
Inactive:       887924 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      31297620 kB
SwapTotal:    67111528 kB
SwapFree:     67055520 kB
Dirty:           33196 kB
Writeback:          12 kB
AnonPages:      228912 kB
Mapped:          20940 kB
Slab:            94312 kB
PageTables:      12496 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  6268772 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73988 MiB
End job on node143 at 2011-06-11 01:19:44