Trace number 3377457

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-random (fixed)? (MO) 702.04 701.876

General information on the benchmark

Namerandom/medium/
unif-k7-r89-v80-c7120-S1818856267-075.UNKNOWN.cnf
MD5SUMff76c2a7684df6ebd90c6aa1ff48f56d
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables80
Number of clauses7120
Sum of the clauses size49840
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 57120

Solver Data 

0.30/0.47	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.30/0.47	c INFO: solving HOME/instance-3377457-1305453318.cnf
0.30/0.47	c INFO: computing task features
0.39/0.52	c INFO: computed variable-clause graph statistics
0.39/0.52	c INFO: computed clause balance statistics
0.59/0.79	c INFO: computed variable balance statistics
0.79/0.90	c INFO: computed small-clause counts
0.79/0.91	c INFO: computed Horn-clause counts
0.79/0.91	c INFO: computed variable graph statistics
0.79/0.91	c INFO: collected features for HOME/instance-3377457-1305453318.cnf in 0.44 s
0.79/0.93	c INFO: running gnovelty+2@0 for 97 with 2399 remaining (b = 0.49)
100.95/101.19	c INFO: running gnovelty+2@0 for 97 with 2299 remaining (b = 0.11)
200.47/200.46	c INFO: running adaptg2wsat2009++@0 for 97 with 2200 remaining (b = 0.06)
300.77/300.72	c INFO: running adaptg2wsat2009++@0 for 97 with 2100 remaining (b = 0.04)
401.07/400.98	c INFO: running adaptg2wsat2009++@0 for 97 with 2000 remaining (b = 0.03)
501.33/501.24	c INFO: running gnovelty+2@0 for 97 with 1900 remaining (b = 0.03)
601.61/601.52	c INFO: running FH@0 for 97 with 1800 remaining (b = 0.02)
701.93/701.78	c INFO: running adaptg2wsat2009++@0 for 97 with 1700 remaining (b = 0.02)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3377457-1305453318/watcher-3377457-1305453318 -o /tmp/evaluation-result-3377457-1305453318/solver-3377457-1305453318 -C 2400 -W 1300 -M 7680 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3377457-1305453318.cnf -seed 602071783 -budget 2400 

running on 2 cores: 1,3

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.47 2.55 3.27 4/355 12412
/proc/meminfo: memFree=17115796/32951124 swapFree=67056804/67111528
[pid=12412] ppid=12410 vsize=15388 CPUtime=0 cores=1,3
/proc/12412/stat : 12412 (python) R 12410 12412 10833 0 -1 4202496 812 0 0 0 0 0 0 0 17 0 1 0 656932827 15757312 698 33554432000 4194304 5683644 140734124424288 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/12412/statm: 3847 698 334 364 0 334 0

[startup+0.106265 s]
/proc/loadavg: 1.47 2.55 3.27 4/355 12412
/proc/meminfo: memFree=17115796/32951124 swapFree=67056804/67111528
[pid=12412] ppid=12410 vsize=104616 CPUtime=0.07 cores=1,3
/proc/12412/stat : 12412 (python) R 12410 12412 10833 0 -1 4202496 3153 0 0 0 6 1 0 0 17 0 1 0 656932827 107126784 2828 33554432000 4194304 5683644 140734124424288 18446744073709551615 4871329 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/12412/statm: 26154 2829 847 364 0 1902 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 104616

[startup+0.200252 s]
/proc/loadavg: 1.47 2.55 3.27 4/355 12412
/proc/meminfo: memFree=17115796/32951124 swapFree=67056804/67111528
[pid=12412] ppid=12410 vsize=128864 CPUtime=0.15 cores=1,3
/proc/12412/stat : 12412 (python) R 12410 12412 10833 0 -1 4202496 5593 935 1 0 12 3 0 0 18 0 1 0 656932827 131956736 4020 33554432000 4194304 5683644 140734124424288 18446744073709551615 270613108736 0 2147483391 16781312 2 0 0 0 17 1 0 0 0
/proc/12412/statm: 32216 4020 1011 364 0 2995 0
Current children cumulated CPU time (s) 0.15
Current children cumulated vsize (KiB) 128864

[startup+0.300232 s]
/proc/loadavg: 1.47 2.55 3.27 4/355 12412
/proc/meminfo: memFree=17115796/32951124 swapFree=67056804/67111528
[pid=12412] ppid=12410 vsize=168676 CPUtime=0.22 cores=1,3
/proc/12412/stat : 12412 (python) R 12410 12412 10833 0 -1 4202496 6718 935 1 0 17 5 0 0 17 0 1 0 656932827 172724224 4791 33554432000 4194304 5683644 140734124424288 18446744073709551615 270613108736 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/12412/statm: 42169 4794 1331 364 0 3454 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 168676

[startup+0.700161 s]
/proc/loadavg: 1.47 2.55 3.27 4/355 12412
/proc/meminfo: memFree=17115796/32951124 swapFree=67056804/67111528
[pid=12412] ppid=12410 vsize=279952 CPUtime=0.59 cores=1,3
/proc/12412/stat : 12412 (python) R 12410 12412 10833 0 -1 4202496 9812 935 1 0 51 8 0 0 17 0 1 0 656932827 286670848 7372 33554432000 4194304 5683644 140734124424288 18446744073709551615 270612759803 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/12412/statm: 69988 7372 1826 364 0 5912 0
Current children cumulated CPU time (s) 0.59
Current children cumulated vsize (KiB) 279952

[startup+1.50103 s]
/proc/loadavg: 1.47 2.55 3.27 5/361 12454
/proc/meminfo: memFree=17087804/32951124 swapFree=67056804/67111528
[pid=12412] ppid=12410 vsize=290220 CPUtime=0.82 cores=1,3
/proc/12412/stat : 12412 (python) S 12410 12412 10833 0 -1 4202496 10113 935 1 0 74 8 0 0 20 0 2 0 656932827 297185280 7427 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12412/statm: 72555 7427 1877 364 0 8473 0
[pid=12449] ppid=12412 vsize=279976 CPUtime=0 cores=1,3
/proc/12449/stat : 12449 (python) S 12412 12412 10833 0 -1 4202560 741 0 0 0 0 0 0 0 19 0 1 0 656932920 286695424 5900 33554432000 4194304 5683644 140734124424288 18446744073709551615 270613130767 0 0 16781312 514 14580606932680179711 0 0 17 1 0 0 0
/proc/12449/statm: 69994 5900 354 364 0 5912 0
[pid=12451] ppid=12449 vsize=1018084 CPUtime=0.56 cores=1,3
/proc/12451/stat : 12451 (gnovelty+2) R 12449 12451 12451 0 -1 4202496 551 0 0 0 56 0 0 0 22 0 1 0 656932920 1042518016 286 33554432000 134512640 135663704 4291758544 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12451/statm: 254521 286 113 282 0 254237 0
[pid=12412/tid=12450] ppid=12410 vsize=290220 CPUtime=0 cores=1,3
/proc/12412/task/12450/stat : 12450 (python) S 12410 12412 10833 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 2 0 656932920 297185280 7427 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1.38
Current children cumulated vsize (KiB) 1588280

[startup+3.10074 s]
/proc/loadavg: 1.47 2.55 3.27 5/361 12454
/proc/meminfo: memFree=17085696/32951124 swapFree=67056804/67111528
[pid=12412] ppid=12410 vsize=290220 CPUtime=0.82 cores=1,3
/proc/12412/stat : 12412 (python) S 12410 12412 10833 0 -1 4202496 10113 935 1 0 74 8 0 0 20 0 2 0 656932827 297185280 7427 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12412/statm: 72555 7427 1877 364 0 8473 0
[pid=12449] ppid=12412 vsize=279976 CPUtime=0 cores=1,3
/proc/12449/stat : 12449 (python) S 12412 12412 10833 0 -1 4202560 741 0 0 0 0 0 0 0 18 0 1 0 656932920 286695424 5900 33554432000 4194304 5683644 140734124424288 18446744073709551615 270613130767 0 0 16781312 514 14580606932680147205 0 0 17 1 0 0 0
/proc/12449/statm: 69994 5900 354 364 0 5912 0
[pid=12451] ppid=12449 vsize=1018084 CPUtime=2.16 cores=1,3
/proc/12451/stat : 12451 (gnovelty+2) R 12449 12451 12451 0 -1 4202496 551 0 0 0 216 0 0 0 25 0 1 0 656932920 1042518016 286 33554432000 134512640 135663704 4291758544 18446744073709551615 134544212 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12451/statm: 254521 286 113 282 0 254237 0
[pid=12412/tid=12450] ppid=12410 vsize=290220 CPUtime=0 cores=1,3
/proc/12412/task/12450/stat : 12450 (python) S 12410 12412 10833 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 2 0 656932920 297185280 7427 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2.98
Current children cumulated vsize (KiB) 1588280

[startup+6.30117 s]
/proc/loadavg: 1.67 2.57 3.27 5/361 12454
/proc/meminfo: memFree=17082992/32951124 swapFree=67056804/67111528
[pid=12412] ppid=12410 vsize=290220 CPUtime=0.82 cores=1,3
/proc/12412/stat : 12412 (python) S 12410 12412 10833 0 -1 4202496 10113 935 1 0 74 8 0 0 20 0 2 0 656932827 297185280 7427 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12412/statm: 72555 7427 1877 364 0 8473 0
[pid=12449] ppid=12412 vsize=279976 CPUtime=0.01 cores=1,3
/proc/12449/stat : 12449 (python) S 12412 12412 10833 0 -1 4202560 982 0 0 0 0 1 0 0 18 0 1 0 656932920 286695424 5928 33554432000 4194304 5683644 140734124424288 18446744073709551615 270613130767 0 0 16781312 514 14580606932680147205 0 0 17 1 0 0 0
/proc/12449/statm: 69994 5928 382 364 0 5912 0
[pid=12451] ppid=12449 vsize=1018084 CPUtime=5.32 cores=1,3
/proc/12451/stat : 12451 (gnovelty+2) R 12449 12451 12451 0 -1 4202496 551 0 0 0 531 1 0 0 25 0 1 0 656932920 1042518016 286 33554432000 134512640 135663704 4291758544 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12451/statm: 254521 286 113 282 0 254237 0
[pid=12412/tid=12450] ppid=12410 vsize=290220 CPUtime=0 cores=1,3
/proc/12412/task/12450/stat : 12450 (python) S 12410 12412 10833 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 2 0 656932920 297185280 7427 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 6.15
Current children cumulated vsize (KiB) 1588280

[startup+12.701 s]
/proc/loadavg: 1.86 2.60 3.28 5/361 12454
/proc/meminfo: memFree=17083684/32951124 swapFree=67056804/67111528
[pid=12412] ppid=12410 vsize=290220 CPUtime=0.82 cores=1,3
/proc/12412/stat : 12412 (python) S 12410 12412 10833 0 -1 4202496 10113 935 1 0 74 8 0 0 20 0 2 0 656932827 297185280 7427 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12412/statm: 72555 7427 1877 364 0 8473 0
[pid=12449] ppid=12412 vsize=279976 CPUtime=0.02 cores=1,3
/proc/12449/stat : 12449 (python) S 12412 12412 10833 0 -1 4202560 983 0 0 0 1 1 0 0 18 0 1 0 656932920 286695424 5928 33554432000 4194304 5683644 140734124424288 18446744073709551615 270613130767 0 0 16781312 514 6791586567749075205 0 0 17 1 0 0 0
/proc/12449/statm: 69994 5928 382 364 0 5912 0
[pid=12451] ppid=12449 vsize=1018084 CPUtime=11.7 cores=1,3
/proc/12451/stat : 12451 (gnovelty+2) R 12449 12451 12451 0 -1 4202496 551 0 0 0 1169 1 0 0 25 0 1 0 656932920 1042518016 286 33554432000 134512640 135663704 4291758544 18446744073709551615 134524503 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12451/statm: 254521 286 113 282 0 254237 0
[pid=12412/tid=12450] ppid=12410 vsize=290220 CPUtime=0 cores=1,3
/proc/12412/task/12450/stat : 12450 (python) S 12410 12412 10833 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 2 0 656932920 297185280 7427 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 12.54
Current children cumulated vsize (KiB) 1588280

[startup+25.5008 s]
/proc/loadavg: 2.33 2.66 3.29 5/361 12454
/proc/meminfo: memFree=17085040/32951124 swapFree=67056804/67111528
[pid=12412] ppid=12410 vsize=290220 CPUtime=0.82 cores=1,3
/proc/12412/stat : 12412 (python) S 12410 12412 10833 0 -1 4202496 10113 935 1 0 74 8 0 0 20 0 2 0 656932827 297185280 7427 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12412/statm: 72555 7427 1877 364 0 8473 0
[pid=12449] ppid=12412 vsize=279976 CPUtime=0.05 cores=1,3
/proc/12449/stat : 12449 (python) S 12412 12412 10833 0 -1 4202560 990 0 0 0 2 3 0 0 18 0 1 0 656932920 286695424 5928 33554432000 4194304 5683644 140734124424288 18446744073709551615 270613130767 0 0 16781312 514 6791586567749075205 0 0 17 1 0 0 0
/proc/12449/statm: 69994 5928 382 364 0 5912 0
[pid=12451] ppid=12449 vsize=1018084 CPUtime=24.47 cores=1,3
/proc/12451/stat : 12451 (gnovelty+2) R 12449 12451 12451 0 -1 4202496 551 0 0 0 2446 1 0 0 25 0 1 0 656932920 1042518016 286 33554432000 134512640 135663704 4291758544 18446744073709551615 134524868 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12451/statm: 254521 286 113 282 0 254237 0
[pid=12412/tid=12450] ppid=12410 vsize=290220 CPUtime=0 cores=1,3
/proc/12412/task/12450/stat : 12450 (python) S 12410 12412 10833 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 2 0 656932920 297185280 7427 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 25.34
Current children cumulated vsize (KiB) 1588280

[startup+51.1013 s]
/proc/loadavg: 2.90 2.77 3.30 5/361 12455
/proc/meminfo: memFree=17085556/32951124 swapFree=67056804/67111528
[pid=12412] ppid=12410 vsize=290220 CPUtime=0.82 cores=1,3
/proc/12412/stat : 12412 (python) S 12410 12412 10833 0 -1 4202496 10113 935 1 0 74 8 0 0 20 0 2 0 656932827 297185280 7427 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12412/statm: 72555 7427 1877 364 0 8473 0
[pid=12449] ppid=12412 vsize=279976 CPUtime=0.12 cores=1,3
/proc/12449/stat : 12449 (python) S 12412 12412 10833 0 -1 4202560 1001 0 0 0 6 6 0 0 18 0 1 0 656932920 286695424 5928 33554432000 4194304 5683644 140734124424288 18446744073709551615 270613130767 0 0 16781312 514 6791586567749075205 0 0 17 1 0 0 0
/proc/12449/statm: 69994 5928 382 364 0 5912 0
[pid=12451] ppid=12449 vsize=1018084 CPUtime=50.01 cores=1,3
/proc/12451/stat : 12451 (gnovelty+2) R 12449 12451 12451 0 -1 4202496 551 0 0 0 5000 1 0 0 25 0 1 0 656932920 1042518016 286 33554432000 134512640 135663704 4291758544 18446744073709551615 134525243 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12451/statm: 254521 286 113 282 0 254237 0
[pid=12412/tid=12450] ppid=12410 vsize=290220 CPUtime=0 cores=1,3
/proc/12412/task/12450/stat : 12450 (python) S 12410 12412 10833 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 2 0 656932920 297185280 7427 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 50.95
Current children cumulated vsize (KiB) 1588280

[startup+102.301 s]
/proc/loadavg: 3.60 2.99 3.35 5/377 12472
/proc/meminfo: memFree=17067480/32951124 swapFree=67056804/67111528
[pid=12412] ppid=12410 vsize=300476 CPUtime=0.83 cores=1,3
/proc/12412/stat : 12412 (python) S 12410 12412 10833 0 -1 4202496 11000 935 1 0 75 8 0 0 20 0 3 0 656932827 307687424 7434 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12412/statm: 75119 7434 1880 364 0 11034 0
[pid=12449] ppid=12412 vsize=290220 CPUtime=0.24 cores=1,3
/proc/12449/stat : 12449 (python) S 12412 12412 10833 0 -1 4202560 1159 0 0 0 13 11 0 0 19 0 2 0 656932920 297185280 5966 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/12449/statm: 72555 5966 415 364 0 8473 0
[pid=12451] ppid=12449 vsize=1018084 CPUtime=99.97 cores=1,3
/proc/12451/stat : 12451 (gnovelty+2) T 12449 12451 12451 0 -1 4202496 551 0 0 0 9996 1 0 0 25 0 1 0 656932920 1042518016 286 33554432000 134512640 135663704 4291758544 18446744073709551615 134524894 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/12451/statm: 254521 286 113 282 0 254237 0
[pid=12449/tid=12465] ppid=12412 vsize=290220 CPUtime=0 cores=1,3
/proc/12449/task/12465/stat : 12465 (python) S 12412 12412 10833 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 656942945 297185280 5966 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 514 18446604469192407232 0 0 -1 3 0 0 0
[pid=12466] ppid=12412 vsize=290232 CPUtime=0 cores=1,3
/proc/12466/stat : 12466 (python) S 12412 12412 10833 0 -1 4202560 742 0 0 0 0 0 0 0 18 0 1 0 656942947 297197568 5898 33554432000 4194304 5683644 140734124424288 18446744073709551615 270613130822 0 0 16781312 514 14434787501581238536 0 0 17 3 0 0 0
/proc/12466/statm: 72558 5898 348 364 0 8473 0
[pid=12468] ppid=12466 vsize=1018084 CPUtime=1.1 cores=1,3
/proc/12468/stat : 12468 (gnovelty+2) R 12466 12468 12468 0 -1 4202496 553 0 0 0 110 0 0 0 19 0 1 0 656942947 1042518016 287 33554432000 134512640 135663704 4294672368 18446744073709551615 134525213 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12468/statm: 254521 287 113 282 0 254237 0
[pid=12412/tid=12450] ppid=12410 vsize=300476 CPUtime=0 cores=1,3
/proc/12412/task/12450/stat : 12450 (python) S 12410 12412 10833 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 3 0 656932920 307687424 7434 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
[pid=12412/tid=12467] ppid=12410 vsize=300476 CPUtime=0 cores=1,3
/proc/12412/task/12467/stat : 12467 (python) S 12410 12412 10833 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 3 0 656942947 307687424 7434 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 102.14
Current children cumulated vsize (KiB) 2917096

[startup+162.301 s]
/proc/loadavg: 3.91 3.18 3.39 5/377 12474
/proc/meminfo: memFree=17064212/32951124 swapFree=67056804/67111528
[pid=12412] ppid=12410 vsize=300476 CPUtime=0.83 cores=1,3
/proc/12412/stat : 12412 (python) S 12410 12412 10833 0 -1 4202496 11000 935 1 0 75 8 0 0 20 0 3 0 656932827 307687424 7434 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12412/statm: 75119 7434 1880 364 0 11034 0
[pid=12449] ppid=12412 vsize=290220 CPUtime=0.24 cores=1,3

################
# More data... #
################

[pid=12485] ppid=12412 vsize=300488 CPUtime=0.25 cores=1,3
/proc/12485/stat : 12485 (python) S 12412 12412 10833 0 -1 4202560 1092 0 0 0 14 11 0 0 19 0 2 0 656952874 307699712 5955 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/12485/statm: 75122 5955 400 364 0 11034 0
[pid=12487] ppid=12485 vsize=417056 CPUtime=99.98 cores=1,3
/proc/12487/stat : 12487 (adaptg2wsat2009) T 12485 12487 12487 0 -1 4202496 720 0 0 0 9997 1 0 0 25 0 1 0 656952874 427065344 450 33554432000 134512640 135107411 4288555968 18446744073709551615 134531561 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/12487/statm: 104264 450 59 146 0 103140 0
[pid=12485/tid=12565] ppid=12412 vsize=300488 CPUtime=0 cores=1,3
/proc/12485/task/12565/stat : 12565 (python) S 12412 12412 10833 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 656962898 307699712 5955 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 514 18446604465882499840 0 0 -1 3 0 0 0
[pid=12566] ppid=12412 vsize=310744 CPUtime=0.25 cores=1,3
/proc/12566/stat : 12566 (python) S 12412 12412 10833 0 -1 4202560 1095 0 0 0 12 13 0 0 19 0 2 0 656962899 318201856 5959 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/12566/statm: 77686 5959 400 364 0 13595 0
[pid=12568] ppid=12566 vsize=417056 CPUtime=100.09 cores=1,3
/proc/12568/stat : 12568 (adaptg2wsat2009) T 12566 12568 12568 0 -1 4202496 720 0 0 0 10009 0 0 0 25 0 1 0 656962900 427065344 450 33554432000 134512640 135107411 4290913760 18446744073709551615 134539419 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/12568/statm: 104264 450 59 146 0 103140 0
[pid=12566/tid=12588] ppid=12412 vsize=310744 CPUtime=0 cores=1,3
/proc/12566/task/12588/stat : 12588 (python) S 12412 12412 10833 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 656972924 318201856 5959 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 514 18446604465882498048 0 0 -1 3 0 0 0
[pid=12589] ppid=12412 vsize=321000 CPUtime=0.25 cores=1,3
/proc/12589/stat : 12589 (python) S 12412 12412 10833 0 -1 4202560 1095 0 0 0 13 12 0 0 19 0 2 0 656972926 328704000 5963 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/12589/statm: 80250 5963 400 364 0 16156 0
[pid=12591] ppid=12589 vsize=417056 CPUtime=99.96 cores=1,3
/proc/12591/stat : 12591 (adaptg2wsat2009) T 12589 12591 12591 0 -1 4202496 721 0 0 0 9996 0 0 0 25 0 1 0 656972926 427065344 451 33554432000 134512640 135107411 4293718192 18446744073709551615 134532277 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/12591/statm: 104264 451 59 146 0 103140 0
[pid=12589/tid=12656] ppid=12412 vsize=321000 CPUtime=0 cores=1,3
/proc/12589/task/12656/stat : 12656 (python) S 12412 12412 10833 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 656982950 328704000 5963 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 514 18446604457752912320 0 0 -1 3 0 0 0
[pid=12657] ppid=12412 vsize=331256 CPUtime=0.25 cores=1,3
/proc/12657/stat : 12657 (python) S 12412 12412 10833 0 -1 4202560 1100 0 0 0 14 11 0 0 19 0 2 0 656982952 339206144 5967 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/12657/statm: 82814 5967 400 364 0 18717 0
[pid=12659] ppid=12657 vsize=1018084 CPUtime=100 cores=1,3
/proc/12659/stat : 12659 (gnovelty+2) T 12657 12659 12659 0 -1 4202496 550 0 0 0 10000 0 0 0 25 0 1 0 656982952 1042518016 287 33554432000 134512640 135663704 4291379664 18446744073709551615 134525007 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/12659/statm: 254521 287 113 282 0 254237 0
[pid=12657/tid=12666] ppid=12412 vsize=331256 CPUtime=0 cores=1,3
/proc/12657/task/12666/stat : 12666 (python) S 12412 12412 10833 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 656992978 339206144 5967 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 514 18446604457752913216 0 0 -1 1 0 0 0
[pid=12667] ppid=12412 vsize=341512 CPUtime=0.24 cores=1,3
/proc/12667/stat : 12667 (python) S 12412 12412 10833 0 -1 4202560 1087 0 0 0 14 10 0 0 19 0 2 0 656992979 349708288 5971 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/12667/statm: 85378 5971 400 364 0 21278 0
[pid=12669] ppid=12667 vsize=247232 CPUtime=100.11 cores=1,3
/proc/12669/stat : 12669 (FH) T 12667 12669 12669 0 -1 4202496 668 0 0 0 10011 0 0 0 25 0 1 0 656992980 253165568 405 33554432000 134512640 134936988 4290763072 18446744073709551615 134552375 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/12669/statm: 61808 405 41 104 0 61312 0
[pid=12667/tid=12672] ppid=12412 vsize=341512 CPUtime=0 cores=1,3
/proc/12667/task/12672/stat : 12672 (python) S 12412 12412 10833 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657003004 349708288 5971 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 514 18446604471139169408 0 0 -1 3 0 0 0
[pid=12673] ppid=12412 vsize=351768 CPUtime=0 cores=1,3
/proc/12673/stat : 12673 (python) S 12412 12412 10833 0 -1 4202560 778 0 0 0 0 0 0 0 19 0 1 0 657003005 360210432 5924 33554432000 4194304 5683644 140734124424288 18446744073709551615 270613130822 0 0 16781312 514 8095630452330594303 0 0 17 1 0 0 0
/proc/12673/statm: 87942 5924 350 364 0 23839 0
[pid=12675] ppid=12673 vsize=416916 CPUtime=0.03 cores=1,3
/proc/12675/stat : 12675 (adaptg2wsat2009) R 12673 12675 12675 0 -1 4202496 646 0 0 0 3 0 0 0 21 0 1 0 657003006 426921984 379 33554432000 134512640 135107411 4291341360 18446744073709551615 134523496 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12675/statm: 104229 379 49 146 0 103105 0
[pid=12412/tid=12450] ppid=12410 vsize=362012 CPUtime=0 cores=1,3
/proc/12412/task/12450/stat : 12450 (python) S 12410 12412 10833 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 9 0 656932920 370700288 7470 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
[pid=12412/tid=12467] ppid=12410 vsize=362012 CPUtime=0 cores=1,3
/proc/12412/task/12467/stat : 12467 (python) S 12410 12412 10833 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 9 0 656942947 370700288 7470 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
[pid=12412/tid=12486] ppid=12410 vsize=362012 CPUtime=0 cores=1,3
/proc/12412/task/12486/stat : 12486 (python) S 12410 12412 10833 0 -1 4202560 28 935 0 0 0 0 0 0 20 0 9 0 656952874 370700288 7470 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
[pid=12412/tid=12567] ppid=12410 vsize=362012 CPUtime=0 cores=1,3
/proc/12412/task/12567/stat : 12567 (python) S 12410 12412 10833 0 -1 4202560 23 935 0 0 0 0 0 0 20 0 9 0 656962899 370700288 7470 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
[pid=12412/tid=12590] ppid=12410 vsize=362012 CPUtime=0 cores=1,3
/proc/12412/task/12590/stat : 12590 (python) S 12410 12412 10833 0 -1 4202560 32 935 0 0 0 0 0 0 20 0 9 0 656972926 370700288 7470 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
[pid=12412/tid=12658] ppid=12410 vsize=362012 CPUtime=0 cores=1,3
/proc/12412/task/12658/stat : 12658 (python) S 12410 12412 10833 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 9 0 656982952 370700288 7470 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
[pid=12412/tid=12668] ppid=12410 vsize=362012 CPUtime=0 cores=1,3
/proc/12412/task/12668/stat : 12668 (python) S 12410 12412 10833 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 9 0 656992979 370700288 7470 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
[pid=12412/tid=12674] ppid=12410 vsize=362012 CPUtime=0 cores=1,3
/proc/12412/task/12674/stat : 12674 (python) S 12410 12412 10833 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 9 0 657003005 370700288 7470 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 702.04
Current children cumulated vsize (KiB) 7868800

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+701.801 s]
/proc/loadavg: 3.05 3.28 3.39 4/396 12675
/proc/meminfo: memFree=16951036/32951124 swapFree=67056804/67111528
[pid=12412] ppid=12410 vsize=362012 CPUtime=0.92 cores=1,3
/proc/12412/stat : 12412 (python) S 12410 12412 10833 0 -1 4202496 16368 935 1 0 83 9 0 0 19 0 9 0 656932827 370700288 7470 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12412/statm: 90503 7470 1892 364 0 26400 0
[pid=12449] ppid=12412 vsize=290220 CPUtime=0.24 cores=1,3
/proc/12449/stat : 12449 (python) S 12412 12412 10833 0 -1 4202560 1159 0 0 0 13 11 0 0 19 0 2 0 656932920 297185280 5966 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/12449/statm: 72555 5966 415 364 0 8473 0
[pid=12451] ppid=12449 vsize=1018084 CPUtime=99.97 cores=1,3
/proc/12451/stat : 12451 (gnovelty+2) T 12449 12451 12451 0 -1 4202496 551 0 0 0 9996 1 0 0 25 0 1 0 656932920 1042518016 286 33554432000 134512640 135663704 4291758544 18446744073709551615 134524894 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/12451/statm: 254521 286 113 282 0 254237 0
[pid=12449/tid=12465] ppid=12412 vsize=290220 CPUtime=0 cores=1,3
/proc/12449/task/12465/stat : 12465 (python) S 12412 12412 10833 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 656942945 297185280 5966 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 514 18446604469192407232 0 0 -1 3 0 0 0
[pid=12466] ppid=12412 vsize=290232 CPUtime=0.25 cores=1,3
/proc/12466/stat : 12466 (python) S 12412 12412 10833 0 -1 4202560 1079 0 0 0 13 12 0 0 19 0 2 0 656942947 297197568 5951 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/12466/statm: 72558 5951 400 364 0 8473 0
[pid=12468] ppid=12466 vsize=1018084 CPUtime=99.25 cores=1,3
/proc/12468/stat : 12468 (gnovelty+2) T 12466 12468 12468 0 -1 4202496 553 0 0 0 9925 0 0 0 25 0 1 0 656942947 1042518016 287 33554432000 134512640 135663704 4294672368 18446744073709551615 134525224 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/12468/statm: 254521 287 113 282 0 254237 0
[pid=12466/tid=12484] ppid=12412 vsize=290232 CPUtime=0 cores=1,3
/proc/12466/task/12484/stat : 12484 (python) S 12412 12412 10833 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 656952872 297197568 5951 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 514 18446604464863488704 0 0 -1 3 0 0 0
[pid=12485] ppid=12412 vsize=300488 CPUtime=0.25 cores=1,3
/proc/12485/stat : 12485 (python) S 12412 12412 10833 0 -1 4202560 1092 0 0 0 14 11 0 0 19 0 2 0 656952874 307699712 5955 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/12485/statm: 75122 5955 400 364 0 11034 0
[pid=12487] ppid=12485 vsize=417056 CPUtime=99.98 cores=1,3
/proc/12487/stat : 12487 (adaptg2wsat2009) T 12485 12487 12487 0 -1 4202496 720 0 0 0 9997 1 0 0 25 0 1 0 656952874 427065344 450 33554432000 134512640 135107411 4288555968 18446744073709551615 134531561 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/12487/statm: 104264 450 59 146 0 103140 0
[pid=12485/tid=12565] ppid=12412 vsize=300488 CPUtime=0 cores=1,3
/proc/12485/task/12565/stat : 12565 (python) S 12412 12412 10833 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 656962898 307699712 5955 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 514 18446604465882499840 0 0 -1 3 0 0 0
[pid=12566] ppid=12412 vsize=310744 CPUtime=0.25 cores=1,3
/proc/12566/stat : 12566 (python) S 12412 12412 10833 0 -1 4202560 1095 0 0 0 12 13 0 0 19 0 2 0 656962899 318201856 5959 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/12566/statm: 77686 5959 400 364 0 13595 0
[pid=12568] ppid=12566 vsize=417056 CPUtime=100.09 cores=1,3
/proc/12568/stat : 12568 (adaptg2wsat2009) T 12566 12568 12568 0 -1 4202496 720 0 0 0 10009 0 0 0 25 0 1 0 656962900 427065344 450 33554432000 134512640 135107411 4290913760 18446744073709551615 134539419 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/12568/statm: 104264 450 59 146 0 103140 0
[pid=12566/tid=12588] ppid=12412 vsize=310744 CPUtime=0 cores=1,3
/proc/12566/task/12588/stat : 12588 (python) S 12412 12412 10833 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 656972924 318201856 5959 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 514 18446604465882498048 0 0 -1 3 0 0 0
[pid=12589] ppid=12412 vsize=321000 CPUtime=0.25 cores=1,3
/proc/12589/stat : 12589 (python) S 12412 12412 10833 0 -1 4202560 1095 0 0 0 13 12 0 0 19 0 2 0 656972926 328704000 5963 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/12589/statm: 80250 5963 400 364 0 16156 0
[pid=12591] ppid=12589 vsize=417056 CPUtime=99.96 cores=1,3
/proc/12591/stat : 12591 (adaptg2wsat2009) T 12589 12591 12591 0 -1 4202496 721 0 0 0 9996 0 0 0 25 0 1 0 656972926 427065344 451 33554432000 134512640 135107411 4293718192 18446744073709551615 134532277 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/12591/statm: 104264 451 59 146 0 103140 0
[pid=12589/tid=12656] ppid=12412 vsize=321000 CPUtime=0 cores=1,3
/proc/12589/task/12656/stat : 12656 (python) S 12412 12412 10833 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 656982950 328704000 5963 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 514 18446604457752912320 0 0 -1 3 0 0 0
[pid=12657] ppid=12412 vsize=331256 CPUtime=0.25 cores=1,3
/proc/12657/stat : 12657 (python) S 12412 12412 10833 0 -1 4202560 1100 0 0 0 14 11 0 0 19 0 2 0 656982952 339206144 5967 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/12657/statm: 82814 5967 400 364 0 18717 0
[pid=12659] ppid=12657 vsize=1018084 CPUtime=100 cores=1,3
/proc/12659/stat : 12659 (gnovelty+2) T 12657 12659 12659 0 -1 4202496 550 0 0 0 10000 0 0 0 25 0 1 0 656982952 1042518016 287 33554432000 134512640 135663704 4291379664 18446744073709551615 134525007 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/12659/statm: 254521 287 113 282 0 254237 0
[pid=12657/tid=12666] ppid=12412 vsize=331256 CPUtime=0 cores=1,3
/proc/12657/task/12666/stat : 12666 (python) S 12412 12412 10833 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 656992978 339206144 5967 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 514 18446604457752913216 0 0 -1 1 0 0 0
[pid=12667] ppid=12412 vsize=341512 CPUtime=0.24 cores=1,3
/proc/12667/stat : 12667 (python) S 12412 12412 10833 0 -1 4202560 1087 0 0 0 14 10 0 0 19 0 2 0 656992979 349708288 5971 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/12667/statm: 85378 5971 400 364 0 21278 0
[pid=12669] ppid=12667 vsize=247232 CPUtime=100.11 cores=1,3
/proc/12669/stat : 12669 (FH) T 12667 12669 12669 0 -1 4202496 668 0 0 0 10011 0 0 0 25 0 1 0 656992980 253165568 405 33554432000 134512640 134936988 4290763072 18446744073709551615 134552375 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/12669/statm: 61808 405 41 104 0 61312 0
[pid=12667/tid=12672] ppid=12412 vsize=341512 CPUtime=0 cores=1,3
/proc/12667/task/12672/stat : 12672 (python) S 12412 12412 10833 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657003004 349708288 5971 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 514 18446604471139169408 0 0 -1 3 0 0 0
[pid=12673] ppid=12412 vsize=351768 CPUtime=0 cores=1,3
/proc/12673/stat : 12673 (python) S 12412 12412 10833 0 -1 4202560 778 0 0 0 0 0 0 0 19 0 1 0 657003005 360210432 5924 33554432000 4194304 5683644 140734124424288 18446744073709551615 270613130822 0 0 16781312 514 8095630452330594303 0 0 17 1 0 0 0
/proc/12673/statm: 87942 5924 350 364 0 23839 0
[pid=12675] ppid=12673 vsize=416916 CPUtime=0.03 cores=1,3
/proc/12675/stat : 12675 (adaptg2wsat2009) R 12673 12675 12675 0 -1 4202496 646 0 0 0 3 0 0 0 21 0 1 0 657003006 426921984 379 33554432000 134512640 135107411 4291341360 18446744073709551615 134523496 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12675/statm: 104229 379 49 146 0 103105 0
[pid=12412/tid=12450] ppid=12410 vsize=362012 CPUtime=0 cores=1,3
/proc/12412/task/12450/stat : 12450 (python) S 12410 12412 10833 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 9 0 656932920 370700288 7470 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
[pid=12412/tid=12467] ppid=12410 vsize=362012 CPUtime=0 cores=1,3
/proc/12412/task/12467/stat : 12467 (python) S 12410 12412 10833 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 9 0 656942947 370700288 7470 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
[pid=12412/tid=12486] ppid=12410 vsize=362012 CPUtime=0 cores=1,3
/proc/12412/task/12486/stat : 12486 (python) S 12410 12412 10833 0 -1 4202560 28 935 0 0 0 0 0 0 20 0 9 0 656952874 370700288 7470 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
[pid=12412/tid=12567] ppid=12410 vsize=362012 CPUtime=0 cores=1,3
/proc/12412/task/12567/stat : 12567 (python) S 12410 12412 10833 0 -1 4202560 23 935 0 0 0 0 0 0 20 0 9 0 656962899 370700288 7470 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
[pid=12412/tid=12590] ppid=12410 vsize=362012 CPUtime=0 cores=1,3
/proc/12412/task/12590/stat : 12590 (python) S 12410 12412 10833 0 -1 4202560 32 935 0 0 0 0 0 0 20 0 9 0 656972926 370700288 7470 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
[pid=12412/tid=12658] ppid=12410 vsize=362012 CPUtime=0 cores=1,3
/proc/12412/task/12658/stat : 12658 (python) S 12410 12412 10833 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 9 0 656982952 370700288 7470 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
[pid=12412/tid=12668] ppid=12410 vsize=362012 CPUtime=0 cores=1,3
/proc/12412/task/12668/stat : 12668 (python) S 12410 12412 10833 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 9 0 656992979 370700288 7470 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
[pid=12412/tid=12674] ppid=12410 vsize=362012 CPUtime=0 cores=1,3
/proc/12412/task/12674/stat : 12674 (python) S 12410 12412 10833 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 9 0 657003005 370700288 7470 33554432000 4194304 5683644 140734124424288 18446744073709551615 270624934321 0 0 16781312 2 18446604454223472448 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 702.04
Current children cumulated vsize (KiB) 7868800

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 12412 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=840872
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=102984
# CPU time returned by wait4() is 0.943856
# while last known CPU time is 702.04
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 701.876
CPU time (s): 702.04
CPU user time (s): 701.13
CPU system time (s): 0.91
CPU usage (%): 100.023
Max. virtual memory (cumulated for all children) (KiB): 7868800

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.840872
system time used= 0.102984
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 17775
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2132
involuntary context switches= 23

runsolver used 2.40863 second user time and 5.20621 second system time

The end

Launcher Data 

Begin job on node108 at 2011-05-15 11:55:19
IDJOB=3377457
IDBENCH=82766
IDSOLVER=1857
FILE ID=node108/3377457-1305453318
RUNJOBID= node108-1305453311-12210
PBS_JOBID= 13324850
Free space on /tmp= 73480 MiB

SOLVER NAME= borg-sat 11.03.02-seq-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k7-r89-v80-c7120-S1818856267-075.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3377457-1305453318/watcher-3377457-1305453318 -o /tmp/evaluation-result-3377457-1305453318/solver-3377457-1305453318 -C 2400 -W 1300 -M 7680  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3377457-1305453318.cnf -seed 602071783 -budget 2400

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= ff76c2a7684df6ebd90c6aa1ff48f56d
RANDOM SEED=602071783

node108.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      17115952 kB
Buffers:        299896 kB
Cached:        5036040 kB
SwapCached:       8700 kB
Active:       11328488 kB
Inactive:      4181912 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      17115952 kB
SwapTotal:    67111528 kB
SwapFree:     67056804 kB
Dirty:          129076 kB
Writeback:           0 kB
AnonPages:    10171704 kB
Mapped:          33900 kB
Slab:           216564 kB
PageTables:      42244 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 54504500 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73480 MiB
End job on node108 at 2011-05-15 12:07:03