Trace number 3484288

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-random (fixed, 1200 s timeout)? 1197.72 1196.27

General information on the benchmark

Namerandom/medium/
unif-k5-r21.3-v150-c3195-S2124915970-046.UNKNOWN.cnf
MD5SUMe6a3a97a5be52c43f493d905174d1bee
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables150
Number of clauses3195
Sum of the clauses size15975
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 53195
Number of clauses of size over 50

Solver Data 

0.36/0.62	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.36/0.62	c INFO: solving HOME/instance-3484288-1307748728.cnf
0.36/0.62	c INFO: computing task features
0.36/0.64	c INFO: computed variable-clause graph statistics
0.36/0.64	c INFO: computed clause balance statistics
0.44/0.72	c INFO: computed variable balance statistics
0.44/0.76	c INFO: computed small-clause counts
0.44/0.76	c INFO: computed Horn-clause counts
0.44/0.77	c INFO: computed variable graph statistics
0.44/0.77	c INFO: collected features for HOME/instance-3484288-1307748728.cnf in 0.14 s
0.44/0.78	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.63)
99.75/99.92	c INFO: running gnovelty+2@0 for 97 with 1100 remaining (b = 0.08)
199.88/200.08	c INFO: running march_hi@0 for 679 with 1000 remaining (b = 0.35)
881.62/880.72	c INFO: running march_hi@680 for 97 with 320 remaining (b = 0.06)
982.66/981.69	c INFO: running march_hi@781 for 219 with 219 remaining (b = 0.10)
1197.57/1196.26	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 988)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3484288-1307748728/watcher-3484288-1307748728 -o /tmp/evaluation-result-3484288-1307748728/solver-3484288-1307748728 -C 1200 -W 1300 -M 7680 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3484288-1307748728.cnf -seed 1972007378 -budget 1200 

running on 2 cores: 0,2

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1200 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1230 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.02 4.00 3.36 5/231 16684
/proc/meminfo: memFree=31338176/32950928 swapFree=67050648/67111528
[pid=16684] ppid=16682 vsize=15184 CPUtime=0 cores=0,2
/proc/16684/stat : 16684 (python) S 16682 16684 15638 0 -1 4202496 729 0 0 0 0 0 0 0 25 0 1 0 886471556 15548416 617 33554432000 4194304 5683644 140737296873776 18446744073709551615 251814238208 0 2147483391 16781312 2 18446744071699215550 0 0 17 0 0 0 0
/proc/16684/statm: 3796 617 322 364 0 283 0

[startup+0.10518 s]
/proc/loadavg: 4.02 4.00 3.36 5/231 16684
/proc/meminfo: memFree=31338176/32950928 swapFree=67050648/67111528
[pid=16684] ppid=16682 vsize=94208 CPUtime=0.06 cores=0,2
/proc/16684/stat : 16684 (python) R 16682 16684 15638 0 -1 4202496 2779 0 0 0 5 1 0 0 25 0 1 0 886471556 96468992 2483 33554432000 4194304 5683644 140737296873776 18446744073709551615 251814238208 0 2147483391 16781312 2 0 0 0 17 2 0 0 0
/proc/16684/statm: 23552 2483 765 364 0 1684 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 94208

[startup+0.200209 s]
/proc/loadavg: 4.02 4.00 3.36 5/231 16684
/proc/meminfo: memFree=31338176/32950928 swapFree=67050648/67111528
[pid=16684] ppid=16682 vsize=126296 CPUtime=0.12 cores=0,2
/proc/16684/stat : 16684 (python) R 16682 16684 15638 0 -1 4202496 4712 933 1 0 9 3 0 0 25 0 1 0 886471556 129327104 3895 33554432000 4194304 5683644 140737296873776 18446744073709551615 251813889081 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/16684/statm: 31574 3895 1001 364 0 2869 0
Current children cumulated CPU time (s) 0.12
Current children cumulated vsize (KiB) 126296

[startup+0.300193 s]
/proc/loadavg: 4.02 4.00 3.36 5/231 16684
/proc/meminfo: memFree=31338176/32950928 swapFree=67050648/67111528
[pid=16684] ppid=16682 vsize=135748 CPUtime=0.17 cores=0,2
/proc/16684/stat : 16684 (python) R 16682 16684 15638 0 -1 4202496 5973 933 1 0 14 3 0 0 25 0 1 0 886471556 139005952 4149 33554432000 4194304 5683644 140737296873776 18446744073709551615 251813895180 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/16684/statm: 33937 4149 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 135748

[startup+0.701122 s]
/proc/loadavg: 4.02 4.00 3.36 5/231 16684
/proc/meminfo: memFree=31338176/32950928 swapFree=67050648/67111528
[pid=16684] ppid=16682 vsize=278764 CPUtime=0.44 cores=0,2
/proc/16684/stat : 16684 (python) R 16682 16684 15638 0 -1 4202496 9409 933 1 0 36 8 0 0 25 0 1 0 886471556 285454336 7066 33554432000 4194304 5683644 140737296873776 18446744073709551615 251817648809 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/16684/statm: 69691 7066 1826 364 0 5615 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 278764

[startup+1.50105 s]
/proc/loadavg: 4.02 4.00 3.36 5/235 16692
/proc/meminfo: memFree=31313616/32950928 swapFree=67050648/67111528
[pid=16684] ppid=16682 vsize=288804 CPUtime=0.52 cores=0,2
/proc/16684/stat : 16684 (python) S 16682 16684 15638 0 -1 4202496 9875 933 1 0 44 8 0 0 25 0 2 0 886471556 295735296 7063 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/16684/statm: 72201 7063 1867 364 0 8119 0
[pid=16690] ppid=16684 vsize=278560 CPUtime=0 cores=0,2
/proc/16690/stat : 16690 (python) S 16684 16684 15638 0 -1 4202560 731 0 0 0 0 0 0 0 25 0 1 0 886471634 285245440 5546 33554432000 4194304 5683644 140737296873776 18446744073709551615 251814260239 0 0 16781312 514 4247205339818098687 0 0 17 2 0 0 0
/proc/16690/statm: 69640 5546 354 364 0 5558 0
[pid=16692] ppid=16690 vsize=1017940 CPUtime=0.71 cores=0,2
/proc/16692/stat : 16692 (gnovelty+2) R 16690 16692 16692 0 -1 4202496 455 0 0 0 71 0 0 0 25 0 1 0 886471634 1042370560 192 33554432000 134512640 135663704 4290564512 18446744073709551615 134524656 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16692/statm: 254485 192 113 282 0 254201 0
[pid=16684/tid=16691] ppid=16682 vsize=288804 CPUtime=0 cores=0,2
/proc/16684/task/16691/stat : 16691 (python) S 16682 16684 15638 0 -1 4202560 30 933 0 0 0 0 0 0 25 0 2 0 886471634 295735296 7063 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1.23
Current children cumulated vsize (KiB) 1585304

[startup+3.10078 s]
/proc/loadavg: 4.02 4.00 3.36 5/235 16692
/proc/meminfo: memFree=31313120/32950928 swapFree=67050648/67111528
[pid=16684] ppid=16682 vsize=288804 CPUtime=0.52 cores=0,2
/proc/16684/stat : 16684 (python) S 16682 16684 15638 0 -1 4202496 9875 933 1 0 44 8 0 0 25 0 2 0 886471556 295735296 7063 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/16684/statm: 72201 7063 1867 364 0 8119 0
[pid=16690] ppid=16684 vsize=278560 CPUtime=0 cores=0,2
/proc/16690/stat : 16690 (python) S 16684 16684 15638 0 -1 4202560 731 0 0 0 0 0 0 0 18 0 1 0 886471634 285245440 5546 33554432000 4194304 5683644 140737296873776 18446744073709551615 251814260239 0 0 16781312 514 4247205339818066180 0 0 17 2 0 0 0
/proc/16690/statm: 69640 5546 354 364 0 5558 0
[pid=16692] ppid=16690 vsize=1017940 CPUtime=2.31 cores=0,2
/proc/16692/stat : 16692 (gnovelty+2) R 16690 16692 16692 0 -1 4202496 455 0 0 0 231 0 0 0 25 0 1 0 886471634 1042370560 192 33554432000 134512640 135663704 4290564512 18446744073709551615 134524503 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16692/statm: 254485 192 113 282 0 254201 0
[pid=16684/tid=16691] ppid=16682 vsize=288804 CPUtime=0 cores=0,2
/proc/16684/task/16691/stat : 16691 (python) S 16682 16684 15638 0 -1 4202560 30 933 0 0 0 0 0 0 25 0 2 0 886471634 295735296 7063 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2.83
Current children cumulated vsize (KiB) 1585304

[startup+6.30118 s]
/proc/loadavg: 4.02 4.00 3.36 5/235 16692
/proc/meminfo: memFree=31312256/32950928 swapFree=67050648/67111528
[pid=16684] ppid=16682 vsize=288804 CPUtime=0.52 cores=0,2
/proc/16684/stat : 16684 (python) S 16682 16684 15638 0 -1 4202496 9875 933 1 0 44 8 0 0 25 0 2 0 886471556 295735296 7063 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/16684/statm: 72201 7063 1867 364 0 8119 0
[pid=16690] ppid=16684 vsize=278560 CPUtime=0 cores=0,2
/proc/16690/stat : 16690 (python) S 16684 16684 15638 0 -1 4202560 870 0 0 0 0 0 0 0 18 0 1 0 886471634 285245440 5569 33554432000 4194304 5683644 140737296873776 18446744073709551615 251814260239 0 0 16781312 514 4247205339818066180 0 0 17 2 0 0 0
/proc/16690/statm: 69640 5569 377 364 0 5558 0
[pid=16692] ppid=16690 vsize=1017940 CPUtime=5.51 cores=0,2
/proc/16692/stat : 16692 (gnovelty+2) R 16690 16692 16692 0 -1 4202496 455 0 0 0 551 0 0 0 25 0 1 0 886471634 1042370560 192 33554432000 134512640 135663704 4290564512 18446744073709551615 134525276 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16692/statm: 254485 192 113 282 0 254201 0
[pid=16684/tid=16691] ppid=16682 vsize=288804 CPUtime=0 cores=0,2
/proc/16684/task/16691/stat : 16691 (python) S 16682 16684 15638 0 -1 4202560 30 933 0 0 0 0 0 0 25 0 2 0 886471634 295735296 7063 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 6.03
Current children cumulated vsize (KiB) 1585304

[startup+12.7011 s]
/proc/loadavg: 4.02 4.00 3.36 5/235 16692
/proc/meminfo: memFree=31312184/32950928 swapFree=67050648/67111528
[pid=16684] ppid=16682 vsize=288804 CPUtime=0.52 cores=0,2
/proc/16684/stat : 16684 (python) S 16682 16684 15638 0 -1 4202496 9875 933 1 0 44 8 0 0 25 0 2 0 886471556 295735296 7063 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/16684/statm: 72201 7063 1867 364 0 8119 0
[pid=16690] ppid=16684 vsize=278560 CPUtime=0 cores=0,2
/proc/16690/stat : 16690 (python) S 16684 16684 15638 0 -1 4202560 875 0 0 0 0 0 0 0 18 0 1 0 886471634 285245440 5569 33554432000 4194304 5683644 140737296873776 18446744073709551615 251814260239 0 0 16781312 514 5162075151012954372 0 0 17 2 0 0 0
/proc/16690/statm: 69640 5569 377 364 0 5558 0
[pid=16692] ppid=16690 vsize=1017940 CPUtime=11.91 cores=0,2
/proc/16692/stat : 16692 (gnovelty+2) R 16690 16692 16692 0 -1 4202496 455 0 0 0 1191 0 0 0 25 0 1 0 886471634 1042370560 192 33554432000 134512640 135663704 4290564512 18446744073709551615 134545644 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16692/statm: 254485 192 113 282 0 254201 0
[pid=16684/tid=16691] ppid=16682 vsize=288804 CPUtime=0 cores=0,2
/proc/16684/task/16691/stat : 16691 (python) S 16682 16684 15638 0 -1 4202560 30 933 0 0 0 0 0 0 25 0 2 0 886471634 295735296 7063 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 12.43
Current children cumulated vsize (KiB) 1585304

[startup+25.5009 s]
/proc/loadavg: 4.01 4.00 3.37 5/238 16695
/proc/meminfo: memFree=31276688/32950928 swapFree=67050648/67111528
[pid=16684] ppid=16682 vsize=288804 CPUtime=0.52 cores=0,2
/proc/16684/stat : 16684 (python) S 16682 16684 15638 0 -1 4202496 9875 933 1 0 44 8 0 0 25 0 2 0 886471556 295735296 7063 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/16684/statm: 72201 7063 1867 364 0 8119 0
[pid=16690] ppid=16684 vsize=278560 CPUtime=0.02 cores=0,2
/proc/16690/stat : 16690 (python) S 16684 16684 15638 0 -1 4202560 880 0 0 0 1 1 0 0 18 0 1 0 886471634 285245440 5569 33554432000 4194304 5683644 140737296873776 18446744073709551615 251814260239 0 0 16781312 514 5162075151012954372 0 0 17 2 0 0 0
/proc/16690/statm: 69640 5569 377 364 0 5558 0
[pid=16692] ppid=16690 vsize=1017940 CPUtime=24.71 cores=0,2
/proc/16692/stat : 16692 (gnovelty+2) R 16690 16692 16692 0 -1 4202496 455 0 0 0 2471 0 0 0 25 0 1 0 886471634 1042370560 192 33554432000 134512640 135663704 4290564512 18446744073709551615 134525271 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16692/statm: 254485 192 113 282 0 254201 0
[pid=16684/tid=16691] ppid=16682 vsize=288804 CPUtime=0 cores=0,2
/proc/16684/task/16691/stat : 16691 (python) S 16682 16684 15638 0 -1 4202560 30 933 0 0 0 0 0 0 25 0 2 0 886471634 295735296 7063 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 25.25
Current children cumulated vsize (KiB) 1585304

[startup+51.1006 s]
/proc/loadavg: 4.01 4.00 3.39 5/238 16696
/proc/meminfo: memFree=31276096/32950928 swapFree=67050648/67111528
[pid=16684] ppid=16682 vsize=288804 CPUtime=0.52 cores=0,2
/proc/16684/stat : 16684 (python) S 16682 16684 15638 0 -1 4202496 9875 933 1 0 44 8 0 0 25 0 2 0 886471556 295735296 7063 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/16684/statm: 72201 7063 1867 364 0 8119 0
[pid=16690] ppid=16684 vsize=278560 CPUtime=0.06 cores=0,2
/proc/16690/stat : 16690 (python) S 16684 16684 15638 0 -1 4202560 891 0 0 0 3 3 0 0 18 0 1 0 886471634 285245440 5569 33554432000 4194304 5683644 140737296873776 18446744073709551615 251814260239 0 0 16781312 514 5162075151012954372 0 0 17 2 0 0 0
/proc/16690/statm: 69640 5569 377 364 0 5558 0
[pid=16692] ppid=16690 vsize=1017940 CPUtime=50.31 cores=0,2
/proc/16692/stat : 16692 (gnovelty+2) R 16690 16692 16692 0 -1 4202496 455 0 0 0 5031 0 0 0 25 0 1 0 886471634 1042370560 192 33554432000 134512640 135663704 4290564512 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16692/statm: 254485 192 113 282 0 254201 0
[pid=16684/tid=16691] ppid=16682 vsize=288804 CPUtime=0 cores=0,2
/proc/16684/task/16691/stat : 16691 (python) S 16682 16684 15638 0 -1 4202560 30 933 0 0 0 0 0 0 25 0 2 0 886471634 295735296 7063 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 50.89
Current children cumulated vsize (KiB) 1585304

[startup+102.301 s]
/proc/loadavg: 4.00 4.00 3.42 5/250 16710
/proc/meminfo: memFree=31254332/32950928 swapFree=67050648/67111528
[pid=16684] ppid=16682 vsize=299532 CPUtime=0.54 cores=0,2
/proc/16684/stat : 16684 (python) S 16682 16684 15638 0 -1 4202496 10712 933 1 0 45 9 0 0 20 0 3 0 886471556 306720768 7206 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/16684/statm: 74883 7206 1870 364 0 10798 0
[pid=16690] ppid=16684 vsize=288804 CPUtime=0.13 cores=0,2
/proc/16690/stat : 16690 (python) S 16684 16684 15638 0 -1 4202560 1147 0 0 0 7 6 0 0 19 0 2 0 886471634 295735296 5613 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/16690/statm: 72201 5613 416 364 0 8119 0
[pid=16692] ppid=16690 vsize=1017940 CPUtime=99.13 cores=0,2
/proc/16692/stat : 16692 (gnovelty+2) T 16690 16692 16692 0 -1 4202496 455 0 0 0 9913 0 0 0 25 0 1 0 886471634 1042370560 192 33554432000 134512640 135663704 4290564512 18446744073709551615 134525278 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16692/statm: 254485 192 113 282 0 254201 0
[pid=16690/tid=16706] ppid=16684 vsize=288804 CPUtime=0 cores=0,2
/proc/16690/task/16706/stat : 16706 (python) S 16684 16684 15638 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 886481547 295735296 5613 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 514 18446604453901572800 0 0 -1 0 0 0 0
[pid=16707] ppid=16684 vsize=289288 CPUtime=0 cores=0,2
/proc/16707/stat : 16707 (python) S 16684 16684 15638 0 -1 4202560 738 0 0 0 0 0 0 0 18 0 1 0 886481548 296230912 5680 33554432000 4194304 5683644 140737296873776 18446744073709551615 251814260294 0 0 16781312 514 4172234314844438785 0 0 17 0 0 0 0
/proc/16707/statm: 72322 5680 348 364 0 8237 0
[pid=16709] ppid=16707 vsize=1017940 CPUtime=2.37 cores=0,2
/proc/16709/stat : 16709 (gnovelty+2) R 16707 16709 16709 0 -1 4202496 454 0 0 0 237 0 0 0 25 0 1 0 886481549 1042370560 191 33554432000 134512640 135663704 4287058752 18446744073709551615 134525218 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16709/statm: 254485 191 113 282 0 254201 0
[pid=16684/tid=16691] ppid=16682 vsize=299532 CPUtime=0 cores=0,2
/proc/16684/task/16691/stat : 16691 (python) S 16682 16684 15638 0 -1 4202560 30 933 0 0 0 0 0 0 25 0 3 0 886471634 306720768 7206 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 2 0 0 0
[pid=16684/tid=16708] ppid=16682 vsize=299532 CPUtime=0 cores=0,2
/proc/16684/task/16708/stat : 16708 (python) S 16682 16684 15638 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 3 0 886481548 306720768 7206 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 102.17
Current children cumulated vsize (KiB) 2913504

[startup+162.301 s]
/proc/loadavg: 4.00 4.00 3.45 5/251 16712
/proc/meminfo: memFree=31251060/32950928 swapFree=67050648/67111528
[pid=16684] ppid=16682 vsize=299532 CPUtime=0.54 cores=0,2
/proc/16684/stat : 16684 (python) S 16682 16684 15638 0 -1 4202496 10712 933 1 0 45 9 0 0 20 0 3 0 886471556 306720768 7206 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/16684/statm: 74883 7206 1870 364 0 10798 0
[pid=16690] ppid=16684 vsize=288804 CPUtime=0.13 cores=0,2

################
# More data... #
################

/proc/16770/statm: 74807 5641 400 364 0 10719 0
[pid=16772] ppid=16770 vsize=208936 CPUtime=990.31 cores=0,2
/proc/16772/stat : 16772 (march_hi) R 16770 16772 16772 0 -1 4202496 44730 0 0 0 99017 14 0 0 25 0 1 0 886491564 213950464 44459 33554432000 4194304 4955518 140736493049792 18446744073709551615 4224443 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16772/statm: 52234 44459 106 186 0 52047 0
[pid=16770/tid=16934] ppid=16684 vsize=299228 CPUtime=0 cores=0,2
/proc/16770/task/16934/stat : 16934 (python) S 16684 16684 15638 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 886559627 306409472 5641 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 514 18446604452232014528 0 0 -1 2 0 0 0
[pid=16684/tid=16691] ppid=16682 vsize=310012 CPUtime=0 cores=0,2
/proc/16684/task/16691/stat : 16691 (python) S 16682 16684 15638 0 -1 4202560 30 933 0 0 0 0 0 0 25 0 4 0 886471634 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 2 0 0 0
[pid=16684/tid=16708] ppid=16682 vsize=310012 CPUtime=0 cores=0,2
/proc/16684/task/16708/stat : 16708 (python) S 16682 16684 15638 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 4 0 886481548 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 0 0 0 0
[pid=16684/tid=16771] ppid=16682 vsize=310012 CPUtime=0 cores=0,2
/proc/16684/task/16771/stat : 16771 (python) S 16682 16684 15638 0 -1 4202560 30 933 0 0 0 0 0 0 18 0 4 0 886491564 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1191.77
Current children cumulated vsize (KiB) 3432148

[startup+1193.61 s]
/proc/loadavg: 4.05 4.04 3.83 5/223 17182
/proc/meminfo: memFree=31332400/32950928 swapFree=67050648/67111528
[pid=16684] ppid=16682 vsize=310012 CPUtime=0.55 cores=0,2
/proc/16684/stat : 16684 (python) S 16682 16684 15638 0 -1 4202496 12132 933 1 0 46 9 0 0 18 0 4 0 886471556 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/16684/statm: 77503 7273 1872 364 0 13415 0
[pid=16690] ppid=16684 vsize=288804 CPUtime=0.13 cores=0,2
/proc/16690/stat : 16690 (python) S 16684 16684 15638 0 -1 4202560 1147 0 0 0 7 6 0 0 19 0 2 0 886471634 295735296 5613 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/16690/statm: 72201 5613 416 364 0 8119 0
[pid=16692] ppid=16690 vsize=1017940 CPUtime=99.13 cores=0,2
/proc/16692/stat : 16692 (gnovelty+2) T 16690 16692 16692 0 -1 4202496 455 0 0 0 9913 0 0 0 25 0 1 0 886471634 1042370560 192 33554432000 134512640 135663704 4290564512 18446744073709551615 134525278 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16692/statm: 254485 192 113 282 0 254201 0
[pid=16690/tid=16706] ppid=16684 vsize=288804 CPUtime=0 cores=0,2
/proc/16690/task/16706/stat : 16706 (python) S 16684 16684 15638 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 886481547 295735296 5613 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 514 18446604453901572800 0 0 -1 0 0 0 0
[pid=16707] ppid=16684 vsize=289288 CPUtime=0.15 cores=0,2
/proc/16707/stat : 16707 (python) S 16684 16684 15638 0 -1 4202560 1079 0 0 0 9 6 0 0 19 0 2 0 886481548 296230912 5733 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/16707/statm: 72322 5733 400 364 0 8237 0
[pid=16709] ppid=16707 vsize=1017940 CPUtime=100.01 cores=0,2
/proc/16709/stat : 16709 (gnovelty+2) T 16707 16709 16709 0 -1 4202496 454 0 0 0 10001 0 0 0 25 0 1 0 886481549 1042370560 191 33554432000 134512640 135663704 4287058752 18446744073709551615 134524882 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16709/statm: 254485 191 113 282 0 254201 0
[pid=16707/tid=16769] ppid=16684 vsize=289288 CPUtime=0 cores=0,2
/proc/16707/task/16769/stat : 16769 (python) S 16684 16684 15638 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 886491563 296230912 5733 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 514 18446604453884117248 0 0 -1 0 0 0 0
[pid=16770] ppid=16684 vsize=299228 CPUtime=1.49 cores=0,2
/proc/16770/stat : 16770 (python) S 16684 16684 15638 0 -1 4202560 1127 0 0 0 82 67 0 0 15 0 2 0 886491564 306409472 5641 33554432000 4194304 5683644 140737296873776 18446744073709551615 251814260294 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/16770/statm: 74807 5641 400 364 0 10719 0
[pid=16772] ppid=16770 vsize=208936 CPUtime=993.52 cores=0,2
/proc/16772/stat : 16772 (march_hi) R 16770 16772 16772 0 -1 4202496 44842 0 0 0 99338 14 0 0 25 0 1 0 886491564 213950464 44571 33554432000 4194304 4955518 140736493049792 18446744073709551615 4224425 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16772/statm: 52234 44571 106 186 0 52047 0
[pid=16770/tid=16934] ppid=16684 vsize=299228 CPUtime=0 cores=0,2
/proc/16770/task/16934/stat : 16934 (python) S 16684 16684 15638 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 886559627 306409472 5641 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 514 18446604452232014528 0 0 -1 2 0 0 0
[pid=16684/tid=16691] ppid=16682 vsize=310012 CPUtime=0 cores=0,2
/proc/16684/task/16691/stat : 16691 (python) S 16682 16684 15638 0 -1 4202560 30 933 0 0 0 0 0 0 25 0 4 0 886471634 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 2 0 0 0
[pid=16684/tid=16708] ppid=16682 vsize=310012 CPUtime=0 cores=0,2
/proc/16684/task/16708/stat : 16708 (python) S 16682 16684 15638 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 4 0 886481548 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 0 0 0 0
[pid=16684/tid=16771] ppid=16682 vsize=310012 CPUtime=0 cores=0,2
/proc/16684/task/16771/stat : 16771 (python) S 16682 16684 15638 0 -1 4202560 30 933 0 0 0 0 0 0 18 0 4 0 886491564 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1194.98
Current children cumulated vsize (KiB) 3432148

[startup+1195.2 s]
/proc/loadavg: 4.05 4.04 3.83 5/223 17182
/proc/meminfo: memFree=31332028/32950928 swapFree=67050648/67111528
[pid=16684] ppid=16682 vsize=310012 CPUtime=0.55 cores=0,2
/proc/16684/stat : 16684 (python) S 16682 16684 15638 0 -1 4202496 12132 933 1 0 46 9 0 0 18 0 4 0 886471556 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/16684/statm: 77503 7273 1872 364 0 13415 0
[pid=16690] ppid=16684 vsize=288804 CPUtime=0.13 cores=0,2
/proc/16690/stat : 16690 (python) S 16684 16684 15638 0 -1 4202560 1147 0 0 0 7 6 0 0 19 0 2 0 886471634 295735296 5613 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/16690/statm: 72201 5613 416 364 0 8119 0
[pid=16692] ppid=16690 vsize=1017940 CPUtime=99.13 cores=0,2
/proc/16692/stat : 16692 (gnovelty+2) T 16690 16692 16692 0 -1 4202496 455 0 0 0 9913 0 0 0 25 0 1 0 886471634 1042370560 192 33554432000 134512640 135663704 4290564512 18446744073709551615 134525278 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16692/statm: 254485 192 113 282 0 254201 0
[pid=16690/tid=16706] ppid=16684 vsize=288804 CPUtime=0 cores=0,2
/proc/16690/task/16706/stat : 16706 (python) S 16684 16684 15638 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 886481547 295735296 5613 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 514 18446604453901572800 0 0 -1 0 0 0 0
[pid=16707] ppid=16684 vsize=289288 CPUtime=0.15 cores=0,2
/proc/16707/stat : 16707 (python) S 16684 16684 15638 0 -1 4202560 1079 0 0 0 9 6 0 0 19 0 2 0 886481548 296230912 5733 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/16707/statm: 72322 5733 400 364 0 8237 0
[pid=16709] ppid=16707 vsize=1017940 CPUtime=100.01 cores=0,2
/proc/16709/stat : 16709 (gnovelty+2) T 16707 16709 16709 0 -1 4202496 454 0 0 0 10001 0 0 0 25 0 1 0 886481549 1042370560 191 33554432000 134512640 135663704 4287058752 18446744073709551615 134524882 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16709/statm: 254485 191 113 282 0 254201 0
[pid=16707/tid=16769] ppid=16684 vsize=289288 CPUtime=0 cores=0,2
/proc/16707/task/16769/stat : 16769 (python) S 16684 16684 15638 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 886491563 296230912 5733 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 514 18446604453884117248 0 0 -1 0 0 0 0
[pid=16770] ppid=16684 vsize=299228 CPUtime=1.49 cores=0,2
/proc/16770/stat : 16770 (python) S 16684 16684 15638 0 -1 4202560 1127 0 0 0 82 67 0 0 15 0 2 0 886491564 306409472 5641 33554432000 4194304 5683644 140737296873776 18446744073709551615 251814260294 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/16770/statm: 74807 5641 400 364 0 10719 0
[pid=16772] ppid=16770 vsize=208936 CPUtime=995.11 cores=0,2
/proc/16772/stat : 16772 (march_hi) R 16770 16772 16772 0 -1 4202496 44907 0 0 0 99497 14 0 0 25 0 1 0 886491564 213950464 44636 33554432000 4194304 4955518 140736493049792 18446744073709551615 4224450 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16772/statm: 52234 44636 106 186 0 52047 0
[pid=16770/tid=16934] ppid=16684 vsize=299228 CPUtime=0 cores=0,2
/proc/16770/task/16934/stat : 16934 (python) S 16684 16684 15638 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 886559627 306409472 5641 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 514 18446604452232014528 0 0 -1 2 0 0 0
[pid=16684/tid=16691] ppid=16682 vsize=310012 CPUtime=0 cores=0,2
/proc/16684/task/16691/stat : 16691 (python) S 16682 16684 15638 0 -1 4202560 30 933 0 0 0 0 0 0 25 0 4 0 886471634 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 2 0 0 0
[pid=16684/tid=16708] ppid=16682 vsize=310012 CPUtime=0 cores=0,2
/proc/16684/task/16708/stat : 16708 (python) S 16682 16684 15638 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 4 0 886481548 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 0 0 0 0
[pid=16684/tid=16771] ppid=16682 vsize=310012 CPUtime=0 cores=0,2
/proc/16684/task/16771/stat : 16771 (python) S 16682 16684 15638 0 -1 4202560 30 933 0 0 0 0 0 0 18 0 4 0 886491564 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1196.57
Current children cumulated vsize (KiB) 3432148

[startup+1196 s]
/proc/loadavg: 4.05 4.04 3.83 5/223 17182
/proc/meminfo: memFree=31331900/32950928 swapFree=67050648/67111528
[pid=16684] ppid=16682 vsize=310012 CPUtime=0.55 cores=0,2
/proc/16684/stat : 16684 (python) S 16682 16684 15638 0 -1 4202496 12132 933 1 0 46 9 0 0 18 0 4 0 886471556 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/16684/statm: 77503 7273 1872 364 0 13415 0
[pid=16690] ppid=16684 vsize=288804 CPUtime=0.13 cores=0,2
/proc/16690/stat : 16690 (python) S 16684 16684 15638 0 -1 4202560 1147 0 0 0 7 6 0 0 19 0 2 0 886471634 295735296 5613 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/16690/statm: 72201 5613 416 364 0 8119 0
[pid=16692] ppid=16690 vsize=1017940 CPUtime=99.13 cores=0,2
/proc/16692/stat : 16692 (gnovelty+2) T 16690 16692 16692 0 -1 4202496 455 0 0 0 9913 0 0 0 25 0 1 0 886471634 1042370560 192 33554432000 134512640 135663704 4290564512 18446744073709551615 134525278 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16692/statm: 254485 192 113 282 0 254201 0
[pid=16690/tid=16706] ppid=16684 vsize=288804 CPUtime=0 cores=0,2
/proc/16690/task/16706/stat : 16706 (python) S 16684 16684 15638 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 886481547 295735296 5613 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 514 18446604453901572800 0 0 -1 0 0 0 0
[pid=16707] ppid=16684 vsize=289288 CPUtime=0.15 cores=0,2
/proc/16707/stat : 16707 (python) S 16684 16684 15638 0 -1 4202560 1079 0 0 0 9 6 0 0 19 0 2 0 886481548 296230912 5733 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/16707/statm: 72322 5733 400 364 0 8237 0
[pid=16709] ppid=16707 vsize=1017940 CPUtime=100.01 cores=0,2
/proc/16709/stat : 16709 (gnovelty+2) T 16707 16709 16709 0 -1 4202496 454 0 0 0 10001 0 0 0 25 0 1 0 886481549 1042370560 191 33554432000 134512640 135663704 4287058752 18446744073709551615 134524882 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16709/statm: 254485 191 113 282 0 254201 0
[pid=16707/tid=16769] ppid=16684 vsize=289288 CPUtime=0 cores=0,2
/proc/16707/task/16769/stat : 16769 (python) S 16684 16684 15638 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 886491563 296230912 5733 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 514 18446604453884117248 0 0 -1 0 0 0 0
[pid=16770] ppid=16684 vsize=299228 CPUtime=1.49 cores=0,2
/proc/16770/stat : 16770 (python) S 16684 16684 15638 0 -1 4202560 1127 0 0 0 82 67 0 0 15 0 2 0 886491564 306409472 5641 33554432000 4194304 5683644 140737296873776 18446744073709551615 251814260294 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/16770/statm: 74807 5641 400 364 0 10719 0
[pid=16772] ppid=16770 vsize=208936 CPUtime=995.91 cores=0,2
/proc/16772/stat : 16772 (march_hi) R 16770 16772 16772 0 -1 4202496 44948 0 0 0 99577 14 0 0 25 0 1 0 886491564 213950464 44677 33554432000 4194304 4955518 140736493049792 18446744073709551615 4208676 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16772/statm: 52234 44677 106 186 0 52047 0
[pid=16770/tid=16934] ppid=16684 vsize=299228 CPUtime=0 cores=0,2
/proc/16770/task/16934/stat : 16934 (python) S 16684 16684 15638 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 886559627 306409472 5641 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 514 18446604452232014528 0 0 -1 2 0 0 0
[pid=16684/tid=16691] ppid=16682 vsize=310012 CPUtime=0 cores=0,2
/proc/16684/task/16691/stat : 16691 (python) S 16682 16684 15638 0 -1 4202560 30 933 0 0 0 0 0 0 25 0 4 0 886471634 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 2 0 0 0
[pid=16684/tid=16708] ppid=16682 vsize=310012 CPUtime=0 cores=0,2
/proc/16684/task/16708/stat : 16708 (python) S 16682 16684 15638 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 4 0 886481548 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 0 0 0 0
[pid=16684/tid=16771] ppid=16682 vsize=310012 CPUtime=0 cores=0,2
/proc/16684/task/16771/stat : 16771 (python) S 16682 16684 15638 0 -1 4202560 30 933 0 0 0 0 0 0 18 0 4 0 886491564 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1197.37
Current children cumulated vsize (KiB) 3432148

[startup+1196.2 s]
/proc/loadavg: 4.05 4.04 3.83 5/223 17182
/proc/meminfo: memFree=31331900/32950928 swapFree=67050648/67111528
[pid=16684] ppid=16682 vsize=310012 CPUtime=0.55 cores=0,2
/proc/16684/stat : 16684 (python) S 16682 16684 15638 0 -1 4202496 12132 933 1 0 46 9 0 0 18 0 4 0 886471556 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/16684/statm: 77503 7273 1872 364 0 13415 0
[pid=16690] ppid=16684 vsize=288804 CPUtime=0.13 cores=0,2
/proc/16690/stat : 16690 (python) S 16684 16684 15638 0 -1 4202560 1147 0 0 0 7 6 0 0 19 0 2 0 886471634 295735296 5613 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/16690/statm: 72201 5613 416 364 0 8119 0
[pid=16692] ppid=16690 vsize=1017940 CPUtime=99.13 cores=0,2
/proc/16692/stat : 16692 (gnovelty+2) T 16690 16692 16692 0 -1 4202496 455 0 0 0 9913 0 0 0 25 0 1 0 886471634 1042370560 192 33554432000 134512640 135663704 4290564512 18446744073709551615 134525278 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16692/statm: 254485 192 113 282 0 254201 0
[pid=16690/tid=16706] ppid=16684 vsize=288804 CPUtime=0 cores=0,2
/proc/16690/task/16706/stat : 16706 (python) S 16684 16684 15638 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 886481547 295735296 5613 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 514 18446604453901572800 0 0 -1 0 0 0 0
[pid=16707] ppid=16684 vsize=289288 CPUtime=0.15 cores=0,2
/proc/16707/stat : 16707 (python) S 16684 16684 15638 0 -1 4202560 1079 0 0 0 9 6 0 0 19 0 2 0 886481548 296230912 5733 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/16707/statm: 72322 5733 400 364 0 8237 0
[pid=16709] ppid=16707 vsize=1017940 CPUtime=100.01 cores=0,2
/proc/16709/stat : 16709 (gnovelty+2) T 16707 16709 16709 0 -1 4202496 454 0 0 0 10001 0 0 0 25 0 1 0 886481549 1042370560 191 33554432000 134512640 135663704 4287058752 18446744073709551615 134524882 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16709/statm: 254485 191 113 282 0 254201 0
[pid=16707/tid=16769] ppid=16684 vsize=289288 CPUtime=0 cores=0,2
/proc/16707/task/16769/stat : 16769 (python) S 16684 16684 15638 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 886491563 296230912 5733 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 514 18446604453884117248 0 0 -1 0 0 0 0
[pid=16770] ppid=16684 vsize=299228 CPUtime=1.49 cores=0,2
/proc/16770/stat : 16770 (python) S 16684 16684 15638 0 -1 4202560 1127 0 0 0 82 67 0 0 15 0 2 0 886491564 306409472 5641 33554432000 4194304 5683644 140737296873776 18446744073709551615 251814260294 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/16770/statm: 74807 5641 400 364 0 10719 0
[pid=16772] ppid=16770 vsize=208936 CPUtime=996.11 cores=0,2
/proc/16772/stat : 16772 (march_hi) R 16770 16772 16772 0 -1 4202496 44957 0 0 0 99597 14 0 0 25 0 1 0 886491564 213950464 44686 33554432000 4194304 4955518 140736493049792 18446744073709551615 4208656 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16772/statm: 52234 44686 106 186 0 52047 0
[pid=16770/tid=16934] ppid=16684 vsize=299228 CPUtime=0 cores=0,2
/proc/16770/task/16934/stat : 16934 (python) S 16684 16684 15638 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 886559627 306409472 5641 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 514 18446604452232014528 0 0 -1 2 0 0 0
[pid=16684/tid=16691] ppid=16682 vsize=310012 CPUtime=0 cores=0,2
/proc/16684/task/16691/stat : 16691 (python) S 16682 16684 15638 0 -1 4202560 30 933 0 0 0 0 0 0 25 0 4 0 886471634 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 2 0 0 0
[pid=16684/tid=16708] ppid=16682 vsize=310012 CPUtime=0 cores=0,2
/proc/16684/task/16708/stat : 16708 (python) S 16682 16684 15638 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 4 0 886481548 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 0 0 0 0
[pid=16684/tid=16771] ppid=16682 vsize=310012 CPUtime=0 cores=0,2
/proc/16684/task/16771/stat : 16771 (python) S 16682 16684 15638 0 -1 4202560 30 933 0 0 0 0 0 0 18 0 4 0 886491564 317452288 7273 33554432000 4194304 5683644 140737296873776 18446744073709551615 251826063793 0 0 16781312 2 18446604453979354432 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1197.57
Current children cumulated vsize (KiB) 3432148

Child status: 0
Real time (s): 1196.27
CPU time (s): 1197.72
CPU user time (s): 1196.64
CPU system time (s): 1.07584
CPU usage (%): 100.121
Max. virtual memory (cumulated for all children) (KiB): 3432148

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1196.64
system time used= 1.07584
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 66406
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4350
involuntary context switches= 1608

runsolver used 2.32365 second user time and 5.03623 second system time

The end

Launcher Data 

Begin job on node126 at 2011-06-11 01:32:08
IDJOB=3484288
IDBENCH=82799
IDSOLVER=1975
FILE ID=node126/3484288-1307748728
RUNJOBID= node126-1307747008-15656
PBS_JOBID= 13550831
Free space on /tmp= 74072 MiB

SOLVER NAME= borg-sat 11.03.02-seq-random (fixed, 1200 s timeout)
BENCH NAME= SAT11/random/medium/unif-k5-r21.3-v150-c3195-S2124915970-046.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3484288-1307748728/watcher-3484288-1307748728 -o /tmp/evaluation-result-3484288-1307748728/solver-3484288-1307748728 -C 1200 -W 1300 -M 7680  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3484288-1307748728.cnf -seed 1972007378 -budget 1200

TIME LIMIT= 1200 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= e6a3a97a5be52c43f493d905174d1bee
RANDOM SEED=1972007378

node126.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      31338604 kB
Buffers:        245396 kB
Cached:         835260 kB
SwapCached:      14888 kB
Active:         844180 kB
Inactive:       556092 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      31338604 kB
SwapTotal:    67111528 kB
SwapFree:     67050648 kB
Dirty:           31908 kB
Writeback:           0 kB
AnonPages:      317332 kB
Mapped:          16740 kB
Slab:           136104 kB
PageTables:      14724 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:  8328264 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74040 MiB
End job on node126 at 2011-06-11 01:52:05