Trace number 3378688

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 3079.79 984.553

General information on the benchmark

Namerandom/medium/
unif-k7-r89-v90-c8010-S1458811765-091.UNKNOWN.cnf
MD5SUM6956d24237a9f7757ce3c5a1a338c9a6
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables90
Number of clauses8010
Sum of the clauses size56070
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 58010

Solver Data 

0.35/0.63	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.35/0.63	c INFO: solving HOME/instance-3378688-1305467036.cnf
0.35/0.63	c INFO: computing task features
0.35/0.68	c INFO: computed variable-clause graph statistics
0.35/0.69	c INFO: computed clause balance statistics
0.64/0.96	c INFO: computed variable balance statistics
0.76/1.07	c INFO: computed small-clause counts
0.76/1.07	c INFO: computed Horn-clause counts
0.76/1.08	c INFO: computed variable graph statistics
0.76/1.08	c INFO: collected features for HOME/instance-3378688-1305467036.cnf in 0.44 s
0.76/1.09	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.47)
0.76/1.10	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.11)
0.76/1.11	c INFO: running adaptg2wsat2009++@0 for 97 with 1199 remaining (b = 0.06)
0.86/1.12	c INFO: running adaptg2wsat2009++@0 for 97 with 1199 remaining (b = 0.05)
300.57/101.45	c INFO: running adaptg2wsat2009++@0 for 97 with 1099 remaining (b = 0.04)
372.92/125.60	c INFO: running gnovelty+2@0 for 97 with 1075 remaining (b = 0.03)
426.67/143.57	c INFO: running FH@0 for 97 with 1057 remaining (b = 0.02)
480.92/161.63	c INFO: running adaptg2wsat2009++@0 for 97 with 1039 remaining (b = 0.02)
727.81/236.92	c INFO: running adaptg2wsat2009++@0 for 97 with 963 remaining (b = 0.02)
731.05/238.07	c INFO: running FH@0 for 97 with 962 remaining (b = 0.01)
810.97/261.00	c INFO: running hybridGM3@0 for 194 with 939 remaining (b = 0.03)
915.24/291.06	c INFO: running gnovelty+2@0 for 97 with 909 remaining (b = 0.01)
1071.84/336.26	c INFO: running hybridGM3@0 for 194 with 864 remaining (b = 0.02)
1293.21/410.69	c INFO: running hybridGM3@0 for 194 with 789 remaining (b = 0.02)
1325.62/421.60	c INFO: running adaptg2wsat2009++@99 for 97 with 779 remaining (b = 0.01)
1591.34/510.84	c INFO: running hybridGM3@100 for 97 with 689 remaining (b = 0.01)
1711.03/551.04	c INFO: running hybridGM3@98 for 97 with 649 remaining (b = 0.01)
1740.79/561.05	c INFO: running march_hi@0 for 388 with 639 remaining (b = 0.04)
2008.17/647.27	c INFO: running hybridGM3@294 for 97 with 553 remaining (b = 0.01)
2078.73/667.60	c INFO: running hybridGM3@98 for 97 with 533 remaining (b = 0.01)
2151.39/688.46	c INFO: running hybridGM3@193 for 97 with 512 remaining (b = 0.01)
2409.62/774.94	c INFO: running FH@0 for 97 with 425 remaining (b = 0.01)
2438.58/784.70	c INFO: running adaptg2wsat2009++@0 for 97 with 415 remaining (b = 0.01)
2703.72/868.03	c INFO: running adaptg2wsat2009++@101 for 97 with 332 remaining (b = 0.01)
2776.35/892.45	c INFO: running adaptg2wsat2009++@98 for 97 with 308 remaining (b = 0.01)
2849.82/917.18	c INFO: running hybridGM3@0 for 283 with 283 remaining (b = 0.02)
3076.81/983.42	c INFO: running gnovelty+2@0 for 97 with 217 remaining (b = 0.01)
3079.79/984.55	c INFO: running hybridGM3@0 for 216 with 216 remaining (b = 0.02)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378688-1305467036/watcher-3378688-1305467036 -o /tmp/evaluation-result-3378688-1305467036/solver-3378688-1305467036 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378688-1305467036.cnf -seed 154170749 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 10.08 10.11 9.67 7/445 30260
/proc/meminfo: memFree=16167088/32951124 swapFree=67082152/67111528
[pid=30260] ppid=30258 vsize=16236 CPUtime=0 cores=1,3,5,7
/proc/30260/stat : 30260 (python) R 30258 30260 28149 0 -1 4202496 995 0 0 0 0 0 0 0 25 0 1 0 658301599 16625664 868 33554432000 4194304 5683644 140736188857360 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/30260/statm: 4059 869 368 364 0 546 0

[startup+0.0764511 s]
/proc/loadavg: 10.08 10.11 9.67 7/445 30260
/proc/meminfo: memFree=16167088/32951124 swapFree=67082152/67111528
[pid=30260] ppid=30258 vsize=45816 CPUtime=0.05 cores=1,3,5,7
/proc/30260/stat : 30260 (python) R 30258 30260 28149 0 -1 4202496 1928 0 0 0 4 1 0 0 25 0 1 0 658301599 46915584 1701 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244346880 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/30260/statm: 11454 1701 470 364 0 1203 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 45816

[startup+0.100462 s]
/proc/loadavg: 10.08 10.11 9.67 7/445 30260
/proc/meminfo: memFree=16167088/32951124 swapFree=67082152/67111528
[pid=30260] ppid=30258 vsize=93960 CPUtime=0.05 cores=1,3,5,7
/proc/30260/stat : 30260 (python) S 30258 30260 28149 0 -1 4202496 2692 0 0 0 4 1 0 0 17 0 1 0 658301599 96215040 2407 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244346880 0 2147483391 16781312 2 18446744071699199166 0 0 17 3 0 0 0
/proc/30260/statm: 23490 2407 764 364 0 1622 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 93960

[startup+0.30038 s]
/proc/loadavg: 10.08 10.11 9.67 7/445 30260
/proc/meminfo: memFree=16167088/32951124 swapFree=67082152/67111528
[pid=30260] ppid=30258 vsize=135748 CPUtime=0.17 cores=1,3,5,7
/proc/30260/stat : 30260 (python) R 30258 30260 28149 0 -1 4202496 5966 935 1 0 13 4 0 0 17 0 1 0 658301599 139005952 4151 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244344837 0 2147483391 16781312 2 0 0 0 17 7 0 0 0
/proc/30260/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 135748

[startup+0.705347 s]
/proc/loadavg: 10.08 10.11 9.67 7/445 30260
/proc/meminfo: memFree=16167088/32951124 swapFree=67082152/67111528
[pid=30260] ppid=30258 vsize=280232 CPUtime=0.45 cores=1,3,5,7
/proc/30260/stat : 30260 (python) R 30258 30260 28149 0 -1 4202496 9997 935 1 0 35 10 0 0 15 0 1 0 658301599 286957568 7443 33554432000 4194304 5683644 140736188857360 18446744073709551615 228247757744 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/30260/statm: 70058 7443 1826 364 0 5982 0
Current children cumulated CPU time (s) 0.45
Current children cumulated vsize (KiB) 280232

[startup+1.50017 s]
/proc/loadavg: 9.84 10.06 9.66 8/446 30265
/proc/meminfo: memFree=16142572/32951124 swapFree=67082152/67111528
[pid=30260] ppid=30258 vsize=320744 CPUtime=0.86 cores=1,3,5,7
/proc/30260/stat : 30260 (python) S 30258 30260 28149 0 -1 4202496 12920 935 1 0 75 11 0 0 20 0 5 0 658301599 328441856 7377 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256175563 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/30260/statm: 80186 7377 1875 364 0 16095 0
Current children cumulated CPU time (s) 0.86
Current children cumulated vsize (KiB) 320744

[startup+3.10089 s]
/proc/loadavg: 9.84 10.06 9.66 11/458 30277
/proc/meminfo: memFree=16120652/32951124 swapFree=67082152/67111528
[pid=30260] ppid=30258 vsize=320744 CPUtime=0.86 cores=1,3,5,7
/proc/30260/stat : 30260 (python) S 30258 30260 28149 0 -1 4202496 12920 935 1 0 75 11 0 0 20 0 5 0 658301599 328441856 7377 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256175563 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/30260/statm: 80186 7377 1875 364 0 16095 0
[pid=30266] ppid=30260 vsize=279412 CPUtime=0 cores=1,3,5,7
/proc/30266/stat : 30266 (python) S 30260 30260 28149 0 -1 4202560 722 0 0 0 0 0 0 0 18 0 1 0 658301708 286117888 5792 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244368911 0 0 16781312 514 7064056273645306118 0 0 17 3 0 0 0
/proc/30266/statm: 69853 5792 354 364 0 5771 0
[pid=30268] ppid=30266 vsize=1018088 CPUtime=1.08 cores=1,3,5,7
/proc/30268/stat : 30268 (gnovelty+2) R 30266 30268 30268 0 -1 4202496 570 0 0 0 108 0 0 0 22 0 1 0 658301708 1042522112 306 33554432000 134512640 135663704 4290318336 18446744073709551615 134525160 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/30268/statm: 254522 306 113 282 0 254238 0
[pid=30269] ppid=30260 vsize=289988 CPUtime=0 cores=1,3,5,7
/proc/30269/stat : 30269 (python) S 30260 30260 28149 0 -1 4202560 733 0 0 0 0 0 0 0 18 0 1 0 658301709 296947712 5838 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244368966 0 0 16781312 514 4328350672337469701 0 0 17 5 0 0 0
/proc/30269/statm: 72497 5838 348 364 0 8412 0
[pid=30271] ppid=30269 vsize=1018088 CPUtime=1.99 cores=1,3,5,7
/proc/30271/stat : 30271 (gnovelty+2) R 30269 30271 30271 0 -1 4202496 569 0 0 0 199 0 0 0 25 0 1 0 658301709 1042522112 306 33554432000 134512640 135663704 4289269216 18446744073709551615 134524874 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/30271/statm: 254522 306 113 282 0 254238 0
[pid=30272] ppid=30260 vsize=300244 CPUtime=0 cores=1,3,5,7
/proc/30272/stat : 30272 (python) S 30260 30260 28149 0 -1 4202560 732 0 0 0 0 0 0 0 18 0 1 0 658301710 307449856 5842 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244368966 0 0 16781312 514 2815744279668850687 0 0 17 3 0 0 0
/proc/30272/statm: 75061 5842 348 364 0 10973 0
[pid=30276] ppid=30272 vsize=417192 CPUtime=1.97 cores=1,3,5,7
/proc/30276/stat : 30276 (adaptg2wsat2009) R 30272 30276 30276 0 -1 4202496 760 0 0 0 197 0 0 0 25 0 1 0 658301711 427204608 495 33554432000 134512640 135107411 4288649808 18446744073709551615 134532107 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/30276/statm: 104298 495 59 146 0 103174 0
[pid=30274] ppid=30260 vsize=310500 CPUtime=0 cores=1,3,5,7
/proc/30274/stat : 30274 (python) S 30260 30260 28149 0 -1 4202560 750 0 0 0 0 0 0 0 18 0 1 0 658301711 317952000 5844 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244368966 0 0 16781312 514 2815748127959547903 0 0 17 3 0 0 0
/proc/30274/statm: 77625 5844 348 364 0 13534 0
[pid=30277] ppid=30274 vsize=417048 CPUtime=0.9 cores=1,3,5,7
/proc/30277/stat : 30277 (adaptg2wsat2009) R 30274 30277 30277 0 -1 4202496 679 0 0 0 90 0 0 0 25 0 1 0 658301711 427057152 419 33554432000 134512640 135107411 4294836208 18446744073709551615 134523296 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/30277/statm: 104262 419 49 146 0 103138 0
[pid=30260/tid=30267] ppid=30258 vsize=320744 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30267/stat : 30267 (python) S 30258 30260 28149 0 -1 4202560 24 935 0 0 0 0 0 0 18 0 5 0 658301708 328441856 7377 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 3 0 0 0
[pid=30260/tid=30270] ppid=30258 vsize=320744 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30270/stat : 30270 (python) S 30258 30260 28149 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 5 0 658301709 328441856 7377 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 5 0 0 0
[pid=30260/tid=30273] ppid=30258 vsize=320744 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30273/stat : 30273 (python) S 30258 30260 28149 0 -1 4202560 38 935 0 0 0 0 0 0 20 0 5 0 658301710 328441856 7377 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 3 0 0 0
[pid=30260/tid=30275] ppid=30258 vsize=320744 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30275/stat : 30275 (python) S 30258 30260 28149 0 -1 4202560 13 935 0 0 0 0 0 0 21 0 5 0 658301711 328441856 7377 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 6.8
Current children cumulated vsize (KiB) 4371304

[startup+6.30132 s]
/proc/loadavg: 9.84 10.06 9.66 11/458 30278
/proc/meminfo: memFree=16116496/32951124 swapFree=67082152/67111528
[pid=30260] ppid=30258 vsize=320744 CPUtime=0.86 cores=1,3,5,7
/proc/30260/stat : 30260 (python) S 30258 30260 28149 0 -1 4202496 12920 935 1 0 75 11 0 0 20 0 5 0 658301599 328441856 7377 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256175563 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/30260/statm: 80186 7377 1875 364 0 16095 0
[pid=30266] ppid=30260 vsize=279412 CPUtime=0.01 cores=1,3,5,7
/proc/30266/stat : 30266 (python) S 30260 30260 28149 0 -1 4202560 882 0 0 0 1 0 0 0 18 0 1 0 658301708 286117888 5814 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244368911 0 0 16781312 514 7064056273645306118 0 0 17 3 0 0 0
/proc/30266/statm: 69853 5814 376 364 0 5771 0
[pid=30268] ppid=30266 vsize=1018088 CPUtime=2.64 cores=1,3,5,7
/proc/30268/stat : 30268 (gnovelty+2) R 30266 30268 30268 0 -1 4202496 570 0 0 0 264 0 0 0 25 0 1 0 658301708 1042522112 306 33554432000 134512640 135663704 4290318336 18446744073709551615 134525243 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/30268/statm: 254522 306 113 282 0 254238 0
[pid=30269] ppid=30260 vsize=289988 CPUtime=0.01 cores=1,3,5,7
/proc/30269/stat : 30269 (python) S 30260 30260 28149 0 -1 4202560 894 0 0 0 0 1 0 0 18 0 1 0 658301709 296947712 5860 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244368966 0 0 16781312 514 4328350672337469701 0 0 17 5 0 0 0
/proc/30269/statm: 72497 5860 370 364 0 8412 0
[pid=30271] ppid=30269 vsize=1018088 CPUtime=5.13 cores=1,3,5,7
/proc/30271/stat : 30271 (gnovelty+2) R 30269 30271 30271 0 -1 4202496 569 0 0 0 513 0 0 0 25 0 1 0 658301709 1042522112 306 33554432000 134512640 135663704 4289269216 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/30271/statm: 254522 306 113 282 0 254238 0
[pid=30272] ppid=30260 vsize=300244 CPUtime=0.01 cores=1,3,5,7
/proc/30272/stat : 30272 (python) S 30260 30260 28149 0 -1 4202560 893 0 0 0 0 1 0 0 18 0 1 0 658301710 307449856 5864 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244368966 0 0 16781312 514 2815744279668818182 0 0 17 3 0 0 0
/proc/30272/statm: 75061 5864 370 364 0 10973 0
[pid=30276] ppid=30272 vsize=417192 CPUtime=5.15 cores=1,3,5,7
/proc/30276/stat : 30276 (adaptg2wsat2009) R 30272 30276 30276 0 -1 4202496 760 0 0 0 515 0 0 0 25 0 1 0 658301711 427204608 495 33554432000 134512640 135107411 4288649808 18446744073709551615 134539338 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/30276/statm: 104298 495 59 146 0 103174 0
[pid=30274] ppid=30260 vsize=310500 CPUtime=0.02 cores=1,3,5,7
/proc/30274/stat : 30274 (python) S 30260 30260 28149 0 -1 4202560 911 0 0 0 1 1 0 0 18 0 1 0 658301711 317952000 5867 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244368966 0 0 16781312 514 2815748127959515397 0 0 17 3 0 0 0
/proc/30274/statm: 77625 5867 371 364 0 13534 0
[pid=30277] ppid=30274 vsize=417192 CPUtime=2.5 cores=1,3,5,7
/proc/30277/stat : 30277 (adaptg2wsat2009) R 30274 30277 30277 0 -1 4202496 755 0 0 0 250 0 0 0 25 0 1 0 658301711 427204608 495 33554432000 134512640 135107411 4294836208 18446744073709551615 134532347 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/30277/statm: 104298 495 59 146 0 103174 0
[pid=30260/tid=30267] ppid=30258 vsize=320744 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30267/stat : 30267 (python) S 30258 30260 28149 0 -1 4202560 24 935 0 0 0 0 0 0 18 0 5 0 658301708 328441856 7377 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 3 0 0 0
[pid=30260/tid=30270] ppid=30258 vsize=320744 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30270/stat : 30270 (python) S 30258 30260 28149 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 5 0 658301709 328441856 7377 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 5 0 0 0
[pid=30260/tid=30273] ppid=30258 vsize=320744 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30273/stat : 30273 (python) S 30258 30260 28149 0 -1 4202560 38 935 0 0 0 0 0 0 20 0 5 0 658301710 328441856 7377 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 3 0 0 0
[pid=30260/tid=30275] ppid=30258 vsize=320744 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30275/stat : 30275 (python) S 30258 30260 28149 0 -1 4202560 13 935 0 0 0 0 0 0 21 0 5 0 658301711 328441856 7377 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 16.33
Current children cumulated vsize (KiB) 4371448

[startup+12.7012 s]
/proc/loadavg: 10.02 10.09 9.67 11/459 30279
/proc/meminfo: memFree=16110560/32951124 swapFree=67082152/67111528
[pid=30260] ppid=30258 vsize=320744 CPUtime=0.86 cores=1,3,5,7
/proc/30260/stat : 30260 (python) S 30258 30260 28149 0 -1 4202496 12920 935 1 0 75 11 0 0 20 0 5 0 658301599 328441856 7377 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256175563 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/30260/statm: 80186 7377 1875 364 0 16095 0
[pid=30266] ppid=30260 vsize=279412 CPUtime=0.03 cores=1,3,5,7
/proc/30266/stat : 30266 (python) S 30260 30260 28149 0 -1 4202560 886 0 0 0 1 2 0 0 18 0 1 0 658301708 286117888 5815 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244368911 0 0 16781312 514 3428857797545001222 0 0 17 1 0 0 0
/proc/30266/statm: 69853 5815 377 364 0 5771 0
[pid=30268] ppid=30266 vsize=1018088 CPUtime=5.82 cores=1,3,5,7
/proc/30268/stat : 30268 (gnovelty+2) R 30266 30268 30268 0 -1 4202496 570 0 0 0 582 0 0 0 25 0 1 0 658301708 1042522112 306 33554432000 134512640 135663704 4290318336 18446744073709551615 134524509 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/30268/statm: 254522 306 113 282 0 254238 0
[pid=30269] ppid=30260 vsize=289988 CPUtime=0.03 cores=1,3,5,7
/proc/30269/stat : 30269 (python) S 30260 30260 28149 0 -1 4202560 898 0 0 0 1 2 0 0 18 0 1 0 658301709 296947712 5861 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244368966 0 0 16781312 514 3428857797545001221 0 0 17 5 0 0 0
/proc/30269/statm: 72497 5861 371 364 0 8412 0
[pid=30271] ppid=30269 vsize=1018088 CPUtime=11.49 cores=1,3,5,7
/proc/30271/stat : 30271 (gnovelty+2) R 30269 30271 30271 0 -1 4202496 569 0 0 0 1149 0 0 0 25 0 1 0 658301709 1042522112 306 33554432000 134512640 135663704 4289269216 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/30271/statm: 254522 306 113 282 0 254238 0
[pid=30272] ppid=30260 vsize=300244 CPUtime=0.03 cores=1,3,5,7
/proc/30272/stat : 30272 (python) S 30260 30260 28149 0 -1 4202560 901 0 0 0 1 2 0 0 18 0 1 0 658301710 307449856 5865 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244368966 0 0 16781312 514 3428857797545001222 0 0 17 3 0 0 0
/proc/30272/statm: 75061 5865 371 364 0 10973 0
[pid=30276] ppid=30272 vsize=417192 CPUtime=9.53 cores=1,3,5,7
/proc/30276/stat : 30276 (adaptg2wsat2009) R 30272 30276 30276 0 -1 4202496 760 0 0 0 953 0 0 0 25 0 1 0 658301711 427204608 495 33554432000 134512640 135107411 4288649808 18446744073709551615 134532313 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/30276/statm: 104298 495 59 146 0 103174 0
[pid=30274] ppid=30260 vsize=310500 CPUtime=0.03 cores=1,3,5,7
/proc/30274/stat : 30274 (python) S 30260 30260 28149 0 -1 4202560 914 0 0 0 2 1 0 0 18 0 1 0 658301711 317952000 5867 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244368966 0 0 16781312 514 3428857797545001221 0 0 17 3 0 0 0
/proc/30274/statm: 77625 5867 371 364 0 13534 0
[pid=30277] ppid=30274 vsize=417192 CPUtime=7.63 cores=1,3,5,7
/proc/30277/stat : 30277 (adaptg2wsat2009) R 30274 30277 30277 0 -1 4202496 755 0 0 0 763 0 0 0 25 0 1 0 658301711 427204608 495 33554432000 134512640 135107411 4294836208 18446744073709551615 134531428 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/30277/statm: 104298 495 59 146 0 103174 0
[pid=30260/tid=30267] ppid=30258 vsize=320744 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30267/stat : 30267 (python) S 30258 30260 28149 0 -1 4202560 24 935 0 0 0 0 0 0 18 0 5 0 658301708 328441856 7377 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 3 0 0 0
[pid=30260/tid=30270] ppid=30258 vsize=320744 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30270/stat : 30270 (python) S 30258 30260 28149 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 5 0 658301709 328441856 7377 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 5 0 0 0
[pid=30260/tid=30273] ppid=30258 vsize=320744 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30273/stat : 30273 (python) S 30258 30260 28149 0 -1 4202560 38 935 0 0 0 0 0 0 20 0 5 0 658301710 328441856 7377 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 3 0 0 0
[pid=30260/tid=30275] ppid=30258 vsize=320744 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30275/stat : 30275 (python) S 30258 30260 28149 0 -1 4202560 13 935 0 0 0 0 0 0 21 0 5 0 658301711 328441856 7377 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 35.45
Current children cumulated vsize (KiB) 4371448

[startup+25.501 s]
/proc/loadavg: 10.02 10.08 9.67 11/459 30279
/proc/meminfo: memFree=16103100/32951124 swapFree=67082152/67111528
[pid=30260] ppid=30258 vsize=320744 CPUtime=0.86 cores=1,3,5,7

################
# More data... #
################

/proc/30274/stat : 30274 (python) S 30260 30260 28149 0 -1 4202560 1143 0 0 0 45 41 0 0 18 0 2 0 658301711 317952000 5899 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/30274/statm: 77625 5899 400 364 0 13534 0
[pid=30277] ppid=30274 vsize=417192 CPUtime=195.79 cores=1,3,5,7
/proc/30277/stat : 30277 (adaptg2wsat2009) T 30274 30277 30277 0 -1 4202496 755 0 0 0 19579 0 0 0 25 0 1 0 658301711 427204608 495 33554432000 134512640 135107411 4294836208 18446744073709551615 134532229 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/30277/statm: 104298 495 59 146 0 103174 0
[pid=30274/tid=30288] ppid=30260 vsize=310500 CPUtime=0 cores=1,3,5,7
/proc/30274/task/30288/stat : 30288 (python) S 30260 30260 28149 0 -1 4202560 46 0 0 0 0 0 0 0 18 0 2 0 658314159 317952000 5899 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 514 18446604457863252992 0 0 -1 3 0 0 0
[pid=30283] ppid=30260 vsize=320756 CPUtime=0.83 cores=1,3,5,7
/proc/30283/stat : 30283 (python) S 30260 30260 28149 0 -1 4202560 1163 0 0 0 41 42 0 0 18 0 2 0 658311745 328454144 5903 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/30283/statm: 80189 5903 400 364 0 16095 0
[pid=30285] ppid=30283 vsize=417192 CPUtime=197.34 cores=1,3,5,7
/proc/30285/stat : 30285 (adaptg2wsat2009) T 30283 30285 30285 0 -1 4202496 760 0 0 0 19734 0 0 0 25 0 1 0 658311745 427204608 495 33554432000 134512640 135107411 4293818848 18446744073709551615 134532326 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/30285/statm: 104298 495 59 146 0 103174 0
[pid=30283/tid=30303] ppid=30260 vsize=320756 CPUtime=0 cores=1,3,5,7
/proc/30283/task/30303/stat : 30303 (python) S 30260 30260 28149 0 -1 4202560 50 0 0 0 0 0 0 0 18 0 2 0 658325290 328454144 5903 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 514 18446604467864254656 0 0 -1 5 0 0 0
[pid=30289] ppid=30260 vsize=331012 CPUtime=0.36 cores=1,3,5,7
/proc/30289/stat : 30289 (python) S 30260 30260 28149 0 -1 4202560 1107 0 0 0 17 19 0 0 19 0 2 0 658314160 338956288 5907 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/30289/statm: 82753 5907 400 364 0 18656 0
[pid=30291] ppid=30289 vsize=1018088 CPUtime=101.48 cores=1,3,5,7
/proc/30291/stat : 30291 (gnovelty+2) T 30289 30291 30291 0 -1 4202496 567 0 0 0 10148 0 0 0 25 0 1 0 658314160 1042522112 307 33554432000 134512640 135663704 4293592512 18446744073709551615 134524882 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/30291/statm: 254522 307 113 282 0 254238 0
[pid=30289/tid=30307] ppid=30260 vsize=331012 CPUtime=0 cores=1,3,5,7
/proc/30289/task/30307/stat : 30307 (python) S 30260 30260 28149 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 658325406 338956288 5907 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 514 18446604463034051264 0 0 -1 3 0 0 0
[pid=30293] ppid=30260 vsize=341268 CPUtime=0.49 cores=1,3,5,7
/proc/30293/stat : 30293 (python) S 30260 30260 28149 0 -1 4202560 1115 0 0 0 25 24 0 0 19 0 2 0 658315956 349458432 5911 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/30293/statm: 85317 5911 400 364 0 21217 0
[pid=30295] ppid=30293 vsize=247352 CPUtime=100.46 cores=1,3,5,7
/proc/30295/stat : 30295 (FH) T 30293 30295 30295 0 -1 4202496 707 0 0 0 10046 0 0 0 25 0 1 0 658315956 253288448 444 33554432000 134512640 134936988 4289447728 18446744073709551615 134552066 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/30295/statm: 61838 444 41 104 0 61342 0
[pid=30293/tid=30319] ppid=30260 vsize=341268 CPUtime=0 cores=1,3,5,7
/proc/30293/task/30319/stat : 30319 (python) S 30260 30260 28149 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 658330704 349458432 5911 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 514 18446604454934884544 0 0 -1 3 0 0 0
[pid=30298] ppid=30260 vsize=351524 CPUtime=0.65 cores=1,3,5,7
/proc/30298/stat : 30298 (python) S 30260 30260 28149 0 -1 4202560 1165 0 0 0 35 30 0 0 18 0 2 0 658317763 359960576 5921 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244368966 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/30298/statm: 87881 5921 406 364 0 23778 0
[pid=30300] ppid=30298 vsize=417192 CPUtime=178 cores=1,3,5,7
/proc/30300/stat : 30300 (adaptg2wsat2009) R 30298 30300 30300 0 -1 4202496 766 0 0 0 17800 0 0 0 25 0 1 0 658317763 427204608 495 33554432000 134512640 135107411 4287659712 18446744073709551615 134531246 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/30300/statm: 104298 495 59 146 0 103174 0
[pid=30298/tid=30314] ppid=30260 vsize=351524 CPUtime=0 cores=1,3,5,7
/proc/30298/task/30314/stat : 30314 (python) S 30260 30260 28149 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 658327698 359960576 5921 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 514 18446604461711012992 0 0 -1 3 0 0 0
[pid=30304] ppid=30260 vsize=361780 CPUtime=0.33 cores=1,3,5,7
/proc/30304/stat : 30304 (python) S 30260 30260 28149 0 -1 4202560 1142 0 0 0 18 15 0 0 19 0 2 0 658325291 370462720 5925 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/30304/statm: 90445 5925 406 364 0 26339 0
[pid=30306] ppid=30304 vsize=417192 CPUtime=97.91 cores=1,3,5,7
/proc/30306/stat : 30306 (adaptg2wsat2009) T 30304 30306 30306 0 -1 4202496 764 0 0 0 9791 0 0 0 25 0 1 0 658325292 427204608 496 33554432000 134512640 135107411 4288082288 18446744073709551615 134532107 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/30306/statm: 104298 496 59 146 0 103174 0
[pid=30304/tid=30327] ppid=30260 vsize=361780 CPUtime=0 cores=1,3,5,7
/proc/30304/task/30327/stat : 30327 (python) S 30260 30260 28149 0 -1 4202560 52 0 0 0 0 0 0 0 19 0 2 0 658335224 370462720 5925 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 514 18446604461039284352 0 0 -1 5 0 0 0
[pid=30308] ppid=30260 vsize=372036 CPUtime=0.58 cores=1,3,5,7
/proc/30308/stat : 30308 (python) S 30260 30260 28149 0 -1 4202560 1130 0 0 0 30 28 0 0 19 0 2 0 658325407 380964864 5923 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/30308/statm: 93009 5923 400 364 0 28900 0
[pid=30310] ppid=30308 vsize=247348 CPUtime=98.93 cores=1,3,5,7
/proc/30310/stat : 30310 (FH) T 30308 30310 30310 0 -1 4202496 710 0 0 0 9893 0 0 0 25 0 1 0 658325407 253284352 444 33554432000 134512640 134936988 4289106080 18446744073709551615 134553554 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/30310/statm: 61837 444 41 104 0 61341 0
[pid=30308/tid=30334] ppid=30260 vsize=372036 CPUtime=0 cores=1,3,5,7
/proc/30308/task/30334/stat : 30334 (python) S 30260 30260 28149 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 658342668 380964864 5923 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 514 18446604469704850368 0 0 -1 7 0 0 0
[pid=30315] ppid=30260 vsize=382292 CPUtime=1.71 cores=1,3,5,7
/proc/30315/stat : 30315 (python) S 30260 30260 28149 0 -1 4202560 1173 0 0 0 91 80 0 0 18 0 2 0 658327699 391467008 5927 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/30315/statm: 95573 5927 400 364 0 31461 0
[pid=30317] ppid=30315 vsize=268428 CPUtime=388.51 cores=1,3,5,7
/proc/30317/stat : 30317 (hybridGM3) T 30315 30317 30317 0 -1 4202496 4940 0 0 0 38851 0 0 0 25 0 1 0 658327700 274870272 4673 33554432000 134512640 135204385 4294141680 18446744073709551615 134537297 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/30317/statm: 67107 4673 64 169 0 66937 0
[pid=30315/tid=30360] ppid=30260 vsize=382292 CPUtime=0 cores=1,3,5,7
/proc/30315/task/30360/stat : 30360 (python) S 30260 30260 28149 0 -1 4202560 49 0 0 0 0 0 0 0 18 0 2 0 658352683 391467008 5927 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 514 18446604454934885440 0 0 -1 5 0 0 0
[pid=30320] ppid=30260 vsize=392548 CPUtime=0.43 cores=1,3,5,7
/proc/30320/stat : 30320 (python) S 30260 30260 28149 0 -1 4202560 1157 0 0 0 24 19 0 0 19 0 2 0 658330706 401969152 5961 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/30320/statm: 98137 5961 400 364 0 34022 0
[pid=30322] ppid=30320 vsize=1018088 CPUtime=98.42 cores=1,3,5,7
/proc/30322/stat : 30322 (gnovelty+2) T 30320 30322 30322 0 -1 4202496 578 0 0 0 9842 0 0 0 25 0 1 0 658330706 1042522112 306 33554432000 134512640 135663704 4290338752 18446744073709551615 134525276 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/30322/statm: 254522 306 113 282 0 254238 0
[pid=30320/tid=30339] ppid=30260 vsize=392548 CPUtime=0 cores=1,3,5,7
/proc/30320/task/30339/stat : 30339 (python) S 30260 30260 28149 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 658343758 401969152 5961 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 514 18446604457941706688 0 0 -1 5 0 0 0
[pid=30328] ppid=30260 vsize=402804 CPUtime=1.73 cores=1,3,5,7
/proc/30328/stat : 30328 (python) S 30260 30260 28149 0 -1 4202560 1183 0 0 0 92 81 0 0 18 0 2 0 658335226 412471296 5965 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/30328/statm: 100701 5965 400 364 0 36583 0
[pid=30330] ppid=30328 vsize=268420 CPUtime=391.05 cores=1,3,5,7
/proc/30330/stat : 30330 (hybridGM3) T 30328 30330 30330 0 -1 4202496 4940 0 0 0 39105 0 0 0 25 0 1 0 658335228 274862080 4671 33554432000 134512640 135204385 4286618848 18446744073709551615 134537483 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/30330/statm: 67105 4671 64 169 0 66935 0
[pid=30328/tid=30362] ppid=30260 vsize=402804 CPUtime=0 cores=1,3,5,7
/proc/30328/task/30362/stat : 30362 (python) S 30260 30260 28149 0 -1 4202560 46 0 0 0 0 0 0 0 18 0 2 0 658356702 412471296 5965 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 514 18446604463034049472 0 0 -1 5 0 0 0
[pid=30335] ppid=30260 vsize=413196 CPUtime=1.18 cores=1,3,5,7
/proc/30335/stat : 30335 (python) S 30260 30260 28149 0 -1 4202560 1181 0 0 0 60 58 0 0 18 0 2 0 658342669 423112704 5973 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/30335/statm: 103299 5973 400 364 0 39178 0
[pid=30337] ppid=30335 vsize=268420 CPUtime=291.73 cores=1,3,5,7
/proc/30337/stat : 30337 (hybridGM3) T 30335 30337 30337 0 -1 4202496 4941 0 0 0 29173 0 0 0 25 0 1 0 658342669 274862080 4671 33554432000 134512640 135204385 4289121696 18446744073709551615 134537443 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/30337/statm: 67105 4671 64 169 0 66935 0
[pid=30335/tid=30369] ppid=30260 vsize=413196 CPUtime=0 cores=1,3,5,7
/proc/30335/task/30369/stat : 30369 (python) S 30260 30260 28149 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 658368358 423112704 5973 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 514 18446604470783339456 0 0 -1 5 0 0 0
[pid=30363] ppid=30260 vsize=423608 CPUtime=1.43 cores=1,3,5,7
/proc/30363/stat : 30363 (python) S 30260 30260 28149 0 -1 4202560 1093 0 0 0 74 69 0 0 15 0 1 0 658357705 433774592 5993 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244368966 0 0 16781312 514 3428857797545001220 0 0 17 7 0 0 0
/proc/30363/statm: 105902 5993 378 364 0 41778 0
[pid=30365] ppid=30363 vsize=63300 CPUtime=364.75 cores=1,3,5,7
/proc/30365/stat : 30365 (march_hi) R 30363 30365 30365 0 -1 4202496 12613 0 0 0 36472 3 0 0 25 0 1 0 658357705 64819200 11055 33554432000 4194304 4955518 140737381392560 18446744073709551615 4297477 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/30365/statm: 15825 11055 106 186 0 15638 0
[pid=30373] ppid=30260 vsize=433864 CPUtime=0.38 cores=1,3,5,7
/proc/30373/stat : 30373 (python) S 30260 30260 28149 0 -1 4202560 1198 0 0 0 20 18 0 0 19 0 2 0 658379093 444276736 6027 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/30373/statm: 108466 6027 407 364 0 44339 0
[pid=30375] ppid=30373 vsize=247344 CPUtime=99.19 cores=1,3,5,7
/proc/30375/stat : 30375 (FH) T 30373 30375 30375 0 -1 4202496 719 0 0 0 9919 0 0 0 25 0 1 0 658379094 253280256 443 33554432000 134512640 134936988 4293627936 18446744073709551615 134553261 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/30375/statm: 61836 443 41 104 0 61340 0
[pid=30373/tid=30379] ppid=30260 vsize=433864 CPUtime=0 cores=1,3,5,7
/proc/30373/task/30379/stat : 30379 (python) S 30260 30260 28149 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 658390843 444276736 6027 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 514 18446604470223229056 0 0 -1 1 0 0 0
[pid=30381] ppid=30260 vsize=444120 CPUtime=0.21 cores=1,3,5,7
/proc/30381/stat : 30381 (python) S 30260 30260 28149 0 -1 4202560 1035 0 0 0 10 11 0 0 15 0 1 0 658393317 454778880 6010 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244368966 0 0 16781312 514 3428857797545001222 0 0 17 5 0 0 0
/proc/30381/statm: 111030 6010 378 364 0 46900 0
[pid=30383] ppid=30381 vsize=268424 CPUtime=64.59 cores=1,3,5,7
/proc/30383/stat : 30383 (hybridGM3) R 30381 30383 30383 0 -1 4202496 4952 0 0 0 6459 0 0 0 25 0 1 0 658393320 274866176 4672 33554432000 134512640 135204385 4289385152 18446744073709551615 134536983 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/30383/statm: 67106 4672 64 169 0 66936 0
[pid=30386] ppid=30260 vsize=454376 CPUtime=0 cores=1,3,5,7
/proc/30386/stat : 30386 (python) S 30260 30260 28149 0 -1 4202560 841 0 0 0 0 0 0 0 19 0 1 0 658399942 465281024 5985 33554432000 4194304 5683644 140736188857360 18446744073709551615 228244368966 0 0 16781312 514 10421488971216322559 0 0 17 5 0 0 0
/proc/30386/statm: 113594 5985 349 364 0 49461 0
[pid=30388] ppid=30386 vsize=1018088 CPUtime=0.44 cores=1,3,5,7
/proc/30388/stat : 30388 (gnovelty+2) R 30386 30388 30388 0 -1 4202496 581 0 0 0 44 0 0 0 22 0 1 0 658399952 1042522112 307 33554432000 134512640 135663704 4287268592 18446744073709551615 134525255 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/30388/statm: 254522 307 113 282 0 254238 0
[pid=30260/tid=30267] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30267/stat : 30267 (python) S 30258 30260 28149 0 -1 4202560 24 935 0 0 0 0 0 0 18 0 19 0 658301708 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 3 0 0 0
[pid=30260/tid=30270] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30270/stat : 30270 (python) S 30258 30260 28149 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 19 0 658301709 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 5 0 0 0
[pid=30260/tid=30273] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30273/stat : 30273 (python) S 30258 30260 28149 0 -1 4202560 42 935 0 0 0 0 0 0 18 0 19 0 658301710 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 3 0 0 0
[pid=30260/tid=30275] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30275/stat : 30275 (python) S 30258 30260 28149 0 -1 4202560 17 935 0 0 0 0 0 0 18 0 19 0 658301711 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 3 0 0 0
[pid=30260/tid=30284] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30284/stat : 30284 (python) S 30258 30260 28149 0 -1 4202560 33 935 0 0 0 0 0 0 18 0 19 0 658311745 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 3 0 0 0
[pid=30260/tid=30290] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30290/stat : 30290 (python) S 30258 30260 28149 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 19 0 658314160 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 3 0 0 0
[pid=30260/tid=30294] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30294/stat : 30294 (python) S 30258 30260 28149 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 19 0 658315956 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 3 0 0 0
[pid=30260/tid=30299] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30299/stat : 30299 (python) S 30258 30260 28149 0 -1 4202560 29 935 0 0 0 0 0 0 18 0 19 0 658317763 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 3 0 0 0
[pid=30260/tid=30305] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30305/stat : 30305 (python) S 30258 30260 28149 0 -1 4202560 20 935 0 0 0 0 0 0 20 0 19 0 658325291 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 5 0 0 0
[pid=30260/tid=30309] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30309/stat : 30309 (python) S 30258 30260 28149 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 19 0 658325407 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 5 0 0 0
[pid=30260/tid=30316] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30316/stat : 30316 (python) S 30258 30260 28149 0 -1 4202560 33 935 0 0 0 0 0 0 18 0 19 0 658327700 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 5 0 0 0
[pid=30260/tid=30321] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30321/stat : 30321 (python) S 30258 30260 28149 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 19 0 658330706 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 5 0 0 0
[pid=30260/tid=30329] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30329/stat : 30329 (python) S 30258 30260 28149 0 -1 4202560 30 935 0 0 0 0 0 0 18 0 19 0 658335226 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 5 0 0 0
[pid=30260/tid=30336] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30336/stat : 30336 (python) S 30258 30260 28149 0 -1 4202560 32 935 0 0 0 0 0 0 18 0 19 0 658342669 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 5 0 0 0
[pid=30260/tid=30364] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30364/stat : 30364 (python) S 30258 30260 28149 0 -1 4202560 30 935 0 0 0 0 0 0 20 0 19 0 658357705 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 5 0 0 0
[pid=30260/tid=30374] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30374/stat : 30374 (python) S 30258 30260 28149 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 19 0 658379093 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 5 0 0 0
[pid=30260/tid=30382] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30382/stat : 30382 (python) S 30258 30260 28149 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 19 0 658393317 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 5 0 0 0
[pid=30260/tid=30387] ppid=30258 vsize=464620 CPUtime=0 cores=1,3,5,7
/proc/30260/task/30387/stat : 30387 (python) S 30258 30260 28149 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 19 0 658399942 475770880 7552 33554432000 4194304 5683644 140736188857360 18446744073709551615 228256172465 0 0 16781312 2 18446604470783338560 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3079.79
Current children cumulated vsize (KiB) 16135628

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 30260 and gives
#  childrusage.ru_utime.tv_sec=437
#  childrusage.ru_utime.tv_usec=839438
#  childrusage.ru_stime.tv_sec=6
#  childrusage.ru_stime.tv_usec=477015
# CPU time returned by wait4() is 444.316
# while last known CPU time is 3079.79
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 984.553
CPU time (s): 3079.79
CPU user time (s): 3073.45
CPU system time (s): 6.34
CPU usage (%): 312.811
Max. virtual memory (cumulated for all children) (KiB): 16135628

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 437.839
system time used= 6.47701
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 66626
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 8821
involuntary context switches= 6005

runsolver used 6.90895 second user time and 15.5046 second system time

The end

Launcher Data 

Begin job on node136 at 2011-05-15 15:43:56
IDJOB=3378688
IDBENCH=83097
IDSOLVER=1859
FILE ID=node136/3378688-1305467036
RUNJOBID= node136-1305462567-28168
PBS_JOBID= 13324968
Free space on /tmp= 72972 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k7-r89-v90-c8010-S1458811765-091.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378688-1305467036/watcher-3378688-1305467036 -o /tmp/evaluation-result-3378688-1305467036/solver-3378688-1305467036 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378688-1305467036.cnf -seed 154170749 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 6956d24237a9f7757ce3c5a1a338c9a6
RANDOM SEED=154170749

node136.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      16167128 kB
Buffers:        322552 kB
Cached:        5948848 kB
SwapCached:       2740 kB
Active:       12018872 kB
Inactive:      4399656 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      16167128 kB
SwapTotal:    67111528 kB
SwapFree:     67082152 kB
Dirty:           31836 kB
Writeback:           0 kB
AnonPages:    10146248 kB
Mapped:          41480 kB
Slab:           242148 kB
PageTables:      56172 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 118636332 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72972 MiB
End job on node136 at 2011-05-15 16:00:23