Trace number 3385751

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)UNSAT 2151.56 332.037

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v450-c1917-S1879031150-062.UNKNOWN.cnf
MD5SUMdea41a8eae52b60f8e05be01aec1734d
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark300.017
Satisfiable
(Un)Satisfiability was proved
Number of variables450
Number of clauses1917
Sum of the clauses size5751
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 31917
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.32/0.45	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.32/0.45	c INFO: solving HOME/instance-3385751-1305572965.cnf
0.32/0.45	c INFO: computing task features
0.32/0.46	c INFO: computed variable-clause graph statistics
0.32/0.46	c INFO: computed clause balance statistics
0.32/0.49	c INFO: computed variable balance statistics
0.41/0.52	c INFO: computed small-clause counts
0.41/0.52	c INFO: computed Horn-clause counts
0.41/0.52	c INFO: computed variable graph statistics
0.41/0.52	c INFO: collected features for HOME/instance-3385751-1305572965.cnf in 0.07 s
0.41/0.55	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.56)
0.41/0.58	c INFO: running march_hi@0 for 388 with 4999 remaining (b = 0.37)
0.52/0.61	c INFO: running march_hi@0 for 388 with 4999 remaining (b = 0.32)
0.52/0.64	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.07)
0.52/0.67	c INFO: running march_hi@0 for 388 with 4999 remaining (b = 0.30)
0.62/0.70	c INFO: running march_hi@0 for 485 with 4999 remaining (b = 0.31)
0.62/0.75	c INFO: running march_hi@0 for 388 with 4999 remaining (b = 0.24)
0.62/0.78	c INFO: running march_hi@0 for 97 with 4999 remaining (b = 0.06)
628.80/100.95	c INFO: running march_hi@0 for 485 with 4899 remaining (b = 0.19)
740.18/119.10	c INFO: running march_hi@0 for 485 with 4881 remaining (b = 0.16)
853.93/136.63	c INFO: running march_hi@100 for 97 with 4863 remaining (b = 0.04)
1665.32/258.03	c INFO: running march_hi@199 for 194 with 4742 remaining (b = 0.07)
2151.28/332.01	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385751-1305572965/watcher-3385751-1305572965 -o /tmp/evaluation-result-3385751-1305572965/solver-3385751-1305572965 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385751-1305572965.cnf -seed 511261501 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.62 8.45 9.56 4/477 29969
/proc/meminfo: memFree=16376376/32951124 swapFree=45193380/67111528
[pid=29969] ppid=29967 vsize=16248 CPUtime=0.01 cores=0-7
/proc/29969/stat : 29969 (python) R 29967 29969 28826 0 -1 4202496 1062 0 0 0 1 0 0 0 17 0 1 0 747527147 16637952 935 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284925696 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/29969/statm: 4062 935 381 364 0 549 0

[startup+0.073208 s]
/proc/loadavg: 4.62 8.45 9.56 4/477 29969
/proc/meminfo: memFree=16376376/32951124 swapFree=45193380/67111528
[pid=29969] ppid=29967 vsize=84612 CPUtime=0.06 cores=0-7
/proc/29969/stat : 29969 (python) R 29967 29969 28826 0 -1 4202496 2401 0 0 0 4 2 0 0 17 0 1 0 747527147 86642688 2131 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284587041 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/29969/statm: 21153 2133 707 364 0 1370 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 84612

[startup+0.10022 s]
/proc/loadavg: 4.62 8.45 9.56 4/477 29969
/proc/meminfo: memFree=16376376/32951124 swapFree=45193380/67111528
[pid=29969] ppid=29967 vsize=104620 CPUtime=0.07 cores=0-7
/proc/29969/stat : 29969 (python) R 29967 29969 28826 0 -1 4202496 3142 0 0 0 5 2 0 0 17 0 1 0 747527147 107130880 2822 33554432000 4194304 5683644 140733259638400 18446744073709551615 5049136 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/29969/statm: 26155 2822 847 364 0 1903 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 104620

[startup+0.300167 s]
/proc/loadavg: 4.62 8.45 9.56 4/477 29969
/proc/meminfo: memFree=16376376/32951124 swapFree=45193380/67111528
[pid=29969] ppid=29967 vsize=198920 CPUtime=0.24 cores=0-7
/proc/29969/stat : 29969 (python) R 29967 29969 28826 0 -1 4202496 7022 934 1 0 17 7 0 0 18 0 1 0 747527147 203694080 5053 33554432000 4194304 5683644 140733259638400 18446744073709551615 4299283 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/29969/statm: 49730 5053 1466 364 0 3757 0
Current children cumulated CPU time (s) 0.24
Current children cumulated vsize (KiB) 198920

[startup+0.701063 s]
/proc/loadavg: 4.62 8.45 9.56 4/477 29969
/proc/meminfo: memFree=16376376/32951124 swapFree=45193380/67111528
[pid=29969] ppid=29967 vsize=395880 CPUtime=0.62 cores=0-7
/proc/29969/stat : 29969 (python) R 29967 29969 28826 0 -1 4202496 13327 934 1 0 51 11 0 0 21 0 6 0 747527147 405381120 7405 33554432000 4194304 5683644 140733259638400 18446744073709551615 224288364479 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/29969/statm: 98970 7405 1882 364 0 34876 0
Current children cumulated CPU time (s) 0.62
Current children cumulated vsize (KiB) 395880

[startup+1.50093 s]
/proc/loadavg: 4.62 8.45 9.56 12/502 29998
/proc/meminfo: memFree=16319228/32951124 swapFree=45193380/67111528
[pid=29969] ppid=29967 vsize=426648 CPUtime=0.69 cores=0-7
/proc/29969/stat : 29969 (python) S 29967 29969 28826 0 -1 4202496 15312 934 1 0 58 11 0 0 25 0 9 0 747527147 436887552 7441 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296752587 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/29969/statm: 106662 7441 1889 364 0 42559 0
[pid=29975] ppid=29969 vsize=278604 CPUtime=0 cores=0-7
/proc/29975/stat : 29975 (python) S 29969 29969 28826 0 -1 4202560 733 0 0 0 0 0 0 0 19 0 1 0 747527203 285290496 5565 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945935 0 0 16781312 514 11459941969522851839 0 0 17 1 0 0 0
/proc/29975/statm: 69651 5565 354 364 0 5569 0
[pid=29977] ppid=29975 vsize=1017800 CPUtime=0.93 cores=0-7
/proc/29977/stat : 29977 (gnovelty+2) R 29975 29977 29977 0 -1 4202496 437 0 0 0 93 0 0 0 23 0 1 0 747527203 1042227200 174 33554432000 134512640 135663704 4291483408 18446744073709551615 134525506 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/29977/statm: 254450 174 113 282 0 254166 0
[pid=29978] ppid=29969 vsize=289332 CPUtime=0 cores=0-7
/proc/29978/stat : 29978 (python) S 29969 29969 28826 0 -1 4202560 741 0 0 0 0 0 0 0 18 0 1 0 747527206 296275968 5690 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 4579896942540062979 0 0 17 3 0 0 0
/proc/29978/statm: 72333 5690 348 364 0 8248 0
[pid=29980] ppid=29978 vsize=2028 CPUtime=0.61 cores=0-7
/proc/29980/stat : 29980 (march_hi) R 29978 29980 29980 0 -1 4202496 590 0 0 0 61 0 0 0 25 0 1 0 747527206 2076672 325 33554432000 4194304 4955518 140733355312896 18446744073709551615 4227552 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/29980/statm: 507 325 104 186 0 320 0
[pid=29981] ppid=29969 vsize=365124 CPUtime=0 cores=0-7
/proc/29981/stat : 29981 (python) S 29969 29969 28826 0 -1 4202560 706 0 0 0 0 0 0 0 15 0 1 0 747527209 373886976 5867 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 5607393195806654724 0 0 17 0 0 0 0
/proc/29981/statm: 91281 5867 348 364 0 27193 0
[pid=29983] ppid=29981 vsize=2028 CPUtime=0.87 cores=0-7
/proc/29983/stat : 29983 (march_hi) R 29981 29983 29983 0 -1 4202496 583 0 0 0 87 0 0 0 23 0 1 0 747527209 2076672 330 33554432000 4194304 4955518 140736246084528 18446744073709551615 4298062 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/29983/statm: 507 330 104 186 0 320 0
[pid=29984] ppid=29969 vsize=375380 CPUtime=0 cores=0-7
/proc/29984/stat : 29984 (python) S 29969 29969 28826 0 -1 4202560 711 0 0 0 0 0 0 0 21 0 1 0 747527212 384389120 5828 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 1263243481975881727 0 0 17 1 0 0 0
/proc/29984/statm: 93845 5828 348 364 0 29754 0
[pid=29986] ppid=29984 vsize=1017800 CPUtime=0.82 cores=0-7
/proc/29986/stat : 29986 (gnovelty+2) R 29984 29986 29986 0 -1 4202496 430 0 0 0 82 0 0 0 25 0 1 0 747527212 1042227200 174 33554432000 134512640 135663704 4290844560 18446744073709551615 134544252 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/29986/statm: 254450 174 113 282 0 254166 0
[pid=29987] ppid=29969 vsize=385636 CPUtime=0 cores=0-7
/proc/29987/stat : 29987 (python) S 29969 29969 28826 0 -1 4202560 727 0 0 0 0 0 0 0 17 0 1 0 747527215 394891264 5850 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 7505187210656710914 0 0 17 2 0 0 0
/proc/29987/statm: 96409 5850 348 364 0 32315 0
[pid=29989] ppid=29987 vsize=2028 CPUtime=0.52 cores=0-7
/proc/29989/stat : 29989 (march_hi) R 29987 29989 29989 0 -1 4202496 576 0 0 0 52 0 0 0 25 0 1 0 747527217 2076672 323 33554432000 4194304 4955518 140733277752256 18446744073709551615 4227502 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/29989/statm: 507 323 104 186 0 320 0
[pid=29990] ppid=29969 vsize=395892 CPUtime=0 cores=0-7
/proc/29990/stat : 29990 (python) S 29969 29969 28826 0 -1 4202560 734 0 0 0 0 0 0 0 15 0 1 0 747527218 405393408 5871 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 7356502452255817985 0 0 17 1 0 0 0
/proc/29990/statm: 98973 5871 348 364 0 34876 0
[pid=29992] ppid=29990 vsize=2028 CPUtime=0.26 cores=0-7
/proc/29992/stat : 29992 (march_hi) R 29990 29992 29992 0 -1 4202496 574 0 0 0 26 0 0 0 25 0 1 0 747527222 2076672 315 33554432000 4194304 4955518 140735650732992 18446744073709551615 4209733 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/29992/statm: 507 315 104 186 0 320 0
[pid=29993] ppid=29969 vsize=406148 CPUtime=0 cores=0-7
/proc/29993/stat : 29993 (python) S 29969 29969 28826 0 -1 4202560 741 0 0 0 0 0 0 0 17 0 1 0 747527223 415895552 5875 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 5284704511787499780 0 0 17 1 0 0 0
/proc/29993/statm: 101537 5875 348 364 0 37437 0
[pid=29995] ppid=29993 vsize=2028 CPUtime=0.29 cores=0-7
/proc/29995/stat : 29995 (march_hi) R 29993 29995 29995 0 -1 4202496 576 0 0 0 29 0 0 0 25 0 1 0 747527225 2076672 315 33554432000 4194304 4955518 140734727806448 18446744073709551615 4224648 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/29995/statm: 507 315 104 186 0 320 0
[pid=29996] ppid=29969 vsize=416404 CPUtime=0 cores=0-7
/proc/29996/stat : 29996 (python) S 29969 29969 28826 0 -1 4202560 779 0 0 0 0 0 0 0 24 0 1 0 747527226 426397696 5902 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 18038513002797302020 0 0 17 1 0 0 0
/proc/29996/statm: 104101 5902 354 364 0 39998 0
[pid=29998] ppid=29996 vsize=2028 CPUtime=0.7 cores=0-7
/proc/29998/stat : 29998 (march_hi) R 29996 29998 29998 0 -1 4202496 597 0 0 0 70 0 0 0 25 0 1 0 747527226 2076672 331 33554432000 4194304 4955518 140735913765408 18446744073709551615 4224651 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/29998/statm: 507 331 104 186 0 320 0
[pid=29969/tid=29976] ppid=29967 vsize=426648 CPUtime=0 cores=0-7
/proc/29969/task/29976/stat : 29976 (python) S 29967 29969 28826 0 -1 4202560 26 934 0 0 0 0 0 0 19 0 9 0 747527203 436887552 7441 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 0 0 0 0
[pid=29969/tid=29979] ppid=29967 vsize=426648 CPUtime=0 cores=0-7
/proc/29969/task/29979/stat : 29979 (python) S 29967 29969 28826 0 -1 4202560 23 934 0 0 0 0 0 0 20 0 9 0 747527206 436887552 7441 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 3 0 0 0
[pid=29969/tid=29982] ppid=29967 vsize=426648 CPUtime=0 cores=0-7
/proc/29969/task/29982/stat : 29982 (python) S 29967 29969 28826 0 -1 4202560 75 934 0 0 0 0 0 0 19 0 9 0 747527209 436887552 7441 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 0 0 0 0
[pid=29969/tid=29985] ppid=29967 vsize=426648 CPUtime=0 cores=0-7
/proc/29969/task/29985/stat : 29985 (python) S 29967 29969 28826 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 9 0 747527212 436887552 7441 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 7 0 0 0
[pid=29969/tid=29988] ppid=29967 vsize=426648 CPUtime=0 cores=0-7
/proc/29969/task/29988/stat : 29988 (python) S 29967 29969 28826 0 -1 4202560 35 934 0 0 0 0 0 0 21 0 9 0 747527215 436887552 7441 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 1 0 0 0
[pid=29969/tid=29991] ppid=29967 vsize=426648 CPUtime=0 cores=0-7
/proc/29969/task/29991/stat : 29991 (python) S 29967 29969 28826 0 -1 4202560 30 934 0 0 0 0 0 0 22 0 9 0 747527218 436887552 7441 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 1 0 0 0
[pid=29969/tid=29994] ppid=29967 vsize=426648 CPUtime=0 cores=0-7
/proc/29969/task/29994/stat : 29994 (python) S 29967 29969 28826 0 -1 4202560 30 934 0 0 0 0 0 0 24 0 9 0 747527223 436887552 7441 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 1 0 0 0
[pid=29969/tid=29997] ppid=29967 vsize=426648 CPUtime=0 cores=0-7
/proc/29969/task/29997/stat : 29997 (python) S 29967 29969 28826 0 -1 4202560 36 934 0 0 0 0 0 0 25 0 9 0 747527226 436887552 7441 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 5.69
Current children cumulated vsize (KiB) 5386936

[startup+3.10166 s]
/proc/loadavg: 4.62 8.45 9.56 11/502 29998
/proc/meminfo: memFree=16318236/32951124 swapFree=45193380/67111528
[pid=29969] ppid=29967 vsize=426648 CPUtime=0.69 cores=0-7
/proc/29969/stat : 29969 (python) S 29967 29969 28826 0 -1 4202496 15312 934 1 0 58 11 0 0 25 0 9 0 747527147 436887552 7441 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296752587 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/29969/statm: 106662 7441 1889 364 0 42559 0
[pid=29975] ppid=29969 vsize=278604 CPUtime=0 cores=0-7
/proc/29975/stat : 29975 (python) S 29969 29969 28826 0 -1 4202560 733 0 0 0 0 0 0 0 18 0 1 0 747527203 285290496 5565 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945935 0 0 16781312 514 11459941969522819329 0 0 17 1 0 0 0
/proc/29975/statm: 69651 5565 354 364 0 5569 0
[pid=29977] ppid=29975 vsize=1017800 CPUtime=2.53 cores=0-7
/proc/29977/stat : 29977 (gnovelty+2) R 29975 29977 29977 0 -1 4202496 437 0 0 0 253 0 0 0 25 0 1 0 747527203 1042227200 174 33554432000 134512640 135663704 4291483408 18446744073709551615 134545793 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/29977/statm: 254450 174 113 282 0 254166 0
[pid=29978] ppid=29969 vsize=289332 CPUtime=0 cores=0-7
/proc/29978/stat : 29978 (python) S 29969 29969 28826 0 -1 4202560 745 0 0 0 0 0 0 0 15 0 1 0 747527206 296275968 5690 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 4579896942540062979 0 0 17 3 0 0 0
/proc/29978/statm: 72333 5690 348 364 0 8248 0
[pid=29980] ppid=29978 vsize=2160 CPUtime=1.41 cores=0-7
/proc/29980/stat : 29980 (march_hi) R 29978 29980 29980 0 -1 4202496 602 0 0 0 141 0 0 0 25 0 1 0 747527206 2211840 337 33554432000 4194304 4955518 140733355312896 18446744073709551615 4209688 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/29980/statm: 540 337 104 186 0 353 0
[pid=29981] ppid=29969 vsize=365124 CPUtime=0 cores=0-7
/proc/29981/stat : 29981 (python) S 29969 29969 28826 0 -1 4202560 709 0 0 0 0 0 0 0 15 0 1 0 747527209 373886976 5867 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 4579910136679596292 0 0 17 0 0 0 0
/proc/29981/statm: 91281 5867 348 364 0 27193 0
[pid=29983] ppid=29981 vsize=2160 CPUtime=2.47 cores=0-7
/proc/29983/stat : 29983 (march_hi) R 29981 29983 29983 0 -1 4202496 603 0 0 0 247 0 0 0 25 0 1 0 747527209 2211840 350 33554432000 4194304 4955518 140736246084528 18446744073709551615 4290981 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/29983/statm: 540 350 104 186 0 353 0
[pid=29984] ppid=29969 vsize=375380 CPUtime=0 cores=0-7
/proc/29984/stat : 29984 (python) S 29969 29969 28826 0 -1 4202560 711 0 0 0 0 0 0 0 18 0 1 0 747527212 384389120 5828 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 1263243481975849220 0 0 17 1 0 0 0
/proc/29984/statm: 93845 5828 348 364 0 29754 0
[pid=29986] ppid=29984 vsize=1017800 CPUtime=2.41 cores=0-7
/proc/29986/stat : 29986 (gnovelty+2) R 29984 29986 29986 0 -1 4202496 430 0 0 0 241 0 0 0 25 0 1 0 747527212 1042227200 174 33554432000 134512640 135663704 4290844560 18446744073709551615 134525302 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/29986/statm: 254450 174 113 282 0 254166 0
[pid=29987] ppid=29969 vsize=385636 CPUtime=0 cores=0-7
/proc/29987/stat : 29987 (python) S 29969 29969 28826 0 -1 4202560 729 0 0 0 0 0 0 0 15 0 1 0 747527215 394891264 5850 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 7505187210656710914 0 0 17 2 0 0 0
/proc/29987/statm: 96409 5850 348 364 0 32315 0
[pid=29989] ppid=29987 vsize=2160 CPUtime=1.32 cores=0-7
/proc/29989/stat : 29989 (march_hi) R 29987 29989 29989 0 -1 4202496 587 0 0 0 132 0 0 0 25 0 1 0 747527217 2211840 334 33554432000 4194304 4955518 140733277752256 18446744073709551615 4224631 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/29989/statm: 540 334 104 186 0 353 0
[pid=29990] ppid=29969 vsize=395892 CPUtime=0 cores=0-7
/proc/29990/stat : 29990 (python) S 29969 29969 28826 0 -1 4202560 734 0 0 0 0 0 0 0 15 0 1 0 747527218 405393408 5871 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 7356502452255817985 0 0 17 1 0 0 0
/proc/29990/statm: 98973 5871 348 364 0 34876 0
[pid=29992] ppid=29990 vsize=2160 CPUtime=1.06 cores=0-7
/proc/29992/stat : 29992 (march_hi) R 29990 29992 29992 0 -1 4202496 592 0 0 0 106 0 0 0 25 0 1 0 747527222 2211840 333 33554432000 4194304 4955518 140735650732992 18446744073709551615 4209730 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/29992/statm: 540 333 104 186 0 353 0
[pid=29993] ppid=29969 vsize=406148 CPUtime=0 cores=0-7
/proc/29993/stat : 29993 (python) S 29969 29969 28826 0 -1 4202560 741 0 0 0 0 0 0 0 15 0 1 0 747527223 415895552 5875 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 412796251780645124 0 0 17 1 0 0 0
/proc/29993/statm: 101537 5875 348 364 0 37437 0
[pid=29995] ppid=29993 vsize=2160 CPUtime=1.09 cores=0-7
/proc/29995/stat : 29995 (march_hi) R 29993 29995 29995 0 -1 4202496 594 0 0 0 109 0 0 0 25 0 1 0 747527225 2211840 333 33554432000 4194304 4955518 140734727806448 18446744073709551615 4261761 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/29995/statm: 540 333 104 186 0 353 0
[pid=29996] ppid=29969 vsize=416404 CPUtime=0 cores=0-7
/proc/29996/stat : 29996 (python) S 29969 29969 28826 0 -1 4202560 781 0 0 0 0 0 0 0 15 0 1 0 747527226 426397696 5902 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 18038513002797302020 0 0 17 1 0 0 0
/proc/29996/statm: 104101 5902 354 364 0 39998 0
[pid=29998] ppid=29996 vsize=2160 CPUtime=2.3 cores=0-7
/proc/29998/stat : 29998 (march_hi) R 29996 29998 29998 0 -1 4202496 617 0 0 0 230 0 0 0 25 0 1 0 747527226 2211840 351 33554432000 4194304 4955518 140735913765408 18446744073709551615 4289943 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/29998/statm: 540 351 104 186 0 353 0
[pid=29969/tid=29976] ppid=29967 vsize=426648 CPUtime=0 cores=0-7
/proc/29969/task/29976/stat : 29976 (python) S 29967 29969 28826 0 -1 4202560 26 934 0 0 0 0 0 0 19 0 9 0 747527203 436887552 7441 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 0 0 0 0
[pid=29969/tid=29979] ppid=29967 vsize=426648 CPUtime=0 cores=0-7
/proc/29969/task/29979/stat : 29979 (python) S 29967 29969 28826 0 -1 4202560 23 934 0 0 0 0 0 0 20 0 9 0 747527206 436887552 7441 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 3 0 0 0
[pid=29969/tid=29982] ppid=29967 vsize=426648 CPUtime=0 cores=0-7
/proc/29969/task/29982/stat : 29982 (python) S 29967 29969 28826 0 -1 4202560 75 934 0 0 0 0 0 0 19 0 9 0 747527209 436887552 7441 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 0 0 0 0
[pid=29969/tid=29985] ppid=29967 vsize=426648 CPUtime=0 cores=0-7
/proc/29969/task/29985/stat : 29985 (python) S 29967 29969 28826 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 9 0 747527212 436887552 7441 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 7 0 0 0
[pid=29969/tid=29988] ppid=29967 vsize=426648 CPUtime=0 cores=0-7

################
# More data... #
################

/proc/29977/statm: 254450 174 113 282 0 254166 0
[pid=29975/tid=30001] ppid=29969 vsize=288848 CPUtime=0 cores=0-7
/proc/29975/task/30001/stat : 30001 (python) S 29969 29969 28826 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 747537239 295780352 5626 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 514 18446604451285951616 0 0 -1 4 0 0 0
[pid=29978] ppid=29969 vsize=289332 CPUtime=1.39 cores=0-7
/proc/29978/stat : 29978 (python) S 29969 29969 28826 0 -1 4202560 998 0 0 0 75 64 0 0 15 0 1 0 747527206 296275968 5713 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 4579896942540062979 0 0 17 3 0 0 0
/proc/29978/statm: 72333 5713 371 364 0 8248 0
[pid=29980] ppid=29978 vsize=11060 CPUtime=297.74 cores=0-7
/proc/29980/stat : 29980 (march_hi) R 29978 29980 29980 0 -1 4202496 2567 0 0 0 29773 1 0 0 25 0 1 0 747527206 11325440 2302 33554432000 4194304 4955518 140733355312896 18446744073709551615 4209701 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/29980/statm: 2765 2302 104 186 0 2578 0
[pid=29981] ppid=29969 vsize=365124 CPUtime=3.42 cores=0-7
/proc/29981/stat : 29981 (python) R 29969 29969 28826 0 -1 4202560 803 0 0 0 254 88 0 0 18 0 1 0 747527209 373886976 5890 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296756031 0 0 16781312 514 0 0 0 17 0 0 0 0
/proc/29981/statm: 91281 5890 371 364 0 27193 0
[pid=29983] ppid=29981 vsize=0 CPUtime=304.69 cores=0-7
/proc/29983/stat : 29983 (march_hi) R 29981 29983 29983 0 -1 4202500 2678 0 0 0 30466 3 0 0 25 0 1 0 747527209 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 0 0 0 0
/proc/29983/statm: 0 0 0 0 0 0 0
[pid=29984] ppid=29969 vsize=375380 CPUtime=0.43 cores=0-7
/proc/29984/stat : 29984 (python) S 29969 29969 28826 0 -1 4202560 970 0 0 0 21 22 0 0 19 0 2 0 747527212 384389120 5881 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296752587 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/29984/statm: 93845 5881 400 364 0 29754 0
[pid=29986] ppid=29984 vsize=1017800 CPUtime=100.63 cores=0-7
/proc/29986/stat : 29986 (gnovelty+2) T 29984 29986 29986 0 -1 4202496 430 0 0 0 10063 0 0 0 25 0 1 0 747527212 1042227200 174 33554432000 134512640 135663704 4290844560 18446744073709551615 134525684 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/29986/statm: 254450 174 113 282 0 254166 0
[pid=29984/tid=30005] ppid=29969 vsize=375380 CPUtime=0 cores=0-7
/proc/29984/task/30005/stat : 30005 (python) S 29969 29969 28826 0 -1 4202560 67 0 0 0 0 0 0 0 19 0 2 0 747539055 384389120 5881 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 514 18446604440321913152 0 0 -1 0 0 0 0
[pid=29987] ppid=29969 vsize=385636 CPUtime=1.35 cores=0-7
/proc/29987/stat : 29987 (python) S 29969 29969 28826 0 -1 4202560 817 0 0 0 69 66 0 0 15 0 1 0 747527215 394891264 5873 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 7505187210656710914 0 0 17 2 0 0 0
/proc/29987/statm: 96409 5873 371 364 0 32315 0
[pid=29989] ppid=29987 vsize=11060 CPUtime=263.88 cores=0-7
/proc/29989/stat : 29989 (march_hi) R 29987 29989 29989 0 -1 4202496 2113 0 0 0 26387 1 0 0 25 0 1 0 747527217 11325440 1860 33554432000 4194304 4955518 140733277752256 18446744073709551615 4209669 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/29989/statm: 2765 1860 104 186 0 2578 0
[pid=29990] ppid=29969 vsize=395892 CPUtime=1.34 cores=0-7
/proc/29990/stat : 29990 (python) S 29969 29969 28826 0 -1 4202560 823 0 0 0 72 62 0 0 15 0 1 0 747527218 405393408 5895 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 7356502452255817985 0 0 17 6 0 0 0
/proc/29990/statm: 98973 5895 372 364 0 34876 0
[pid=29992] ppid=29990 vsize=11060 CPUtime=265.67 cores=0-7
/proc/29992/stat : 29992 (march_hi) R 29990 29992 29992 0 -1 4202496 2143 0 0 0 26567 0 0 0 25 0 1 0 747527222 11325440 1884 33554432000 4194304 4955518 140735650732992 18446744073709551615 4227824 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/29992/statm: 2765 1884 104 186 0 2578 0
[pid=29993] ppid=29969 vsize=406148 CPUtime=1.35 cores=0-7
/proc/29993/stat : 29993 (python) S 29969 29969 28826 0 -1 4202560 833 0 0 0 67 68 0 0 15 0 1 0 747527223 415895552 5898 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 412796251780645124 0 0 17 1 0 0 0
/proc/29993/statm: 101537 5898 371 364 0 37437 0
[pid=29995] ppid=29993 vsize=8048 CPUtime=213.01 cores=0-7
/proc/29995/stat : 29995 (march_hi) R 29993 29995 29995 0 -1 4202496 1616 0 0 0 21300 1 0 0 25 0 1 0 747527225 8241152 1355 33554432000 4194304 4955518 140734727806448 18446744073709551615 4227811 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/29995/statm: 2012 1355 104 186 0 1825 0
[pid=29996] ppid=29969 vsize=416404 CPUtime=1.34 cores=0-7
/proc/29996/stat : 29996 (python) S 29969 29969 28826 0 -1 4202560 1065 0 0 0 70 64 0 0 15 0 2 0 747527226 426397696 5955 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/29996/statm: 104101 5955 406 364 0 39998 0
[pid=29998] ppid=29996 vsize=8048 CPUtime=238.31 cores=0-7
/proc/29998/stat : 29998 (march_hi) R 29996 29998 29998 0 -1 4202496 1859 0 0 0 23830 1 0 0 25 0 1 0 747527226 8241152 1593 33554432000 4194304 4955518 140735913765408 18446744073709551615 4295582 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/29998/statm: 2012 1593 104 186 0 1825 0
[pid=29996/tid=30009] ppid=29969 vsize=416404 CPUtime=0 cores=0-7
/proc/29996/task/30009/stat : 30009 (python) S 29969 29969 28826 0 -1 4202560 83 0 0 0 0 0 0 0 15 0 2 0 747540807 426397696 5955 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 514 18446604457272032256 0 0 -1 0 0 0 0
[pid=30002] ppid=29969 vsize=426660 CPUtime=0.93 cores=0-7
/proc/30002/stat : 30002 (python) S 29969 29969 28826 0 -1 4202560 850 0 0 0 50 43 0 0 15 0 1 0 747537243 436899840 5928 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 3960634401590575367 0 0 17 4 0 0 0
/proc/30002/statm: 106665 5928 371 364 0 42559 0
[pid=30004] ppid=30002 vsize=6112 CPUtime=182.1 cores=0-7
/proc/30004/stat : 30004 (march_hi) R 30002 30004 30004 0 -1 4202496 1384 0 0 0 18210 0 0 0 25 0 1 0 747537243 6258688 1118 33554432000 4194304 4955518 140733951866352 18446744073709551615 4346317 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/30004/statm: 1528 1118 104 186 0 1341 0
[pid=30006] ppid=29969 vsize=436916 CPUtime=0.87 cores=0-7
/proc/30006/stat : 30006 (python) S 29969 29969 28826 0 -1 4202560 866 0 0 0 44 43 0 0 16 0 1 0 747539058 447401984 5975 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 18216902163039224068 0 0 17 1 0 0 0
/proc/30006/statm: 109229 5975 371 364 0 45120 0
[pid=30008] ppid=30006 vsize=6112 CPUtime=170.58 cores=0-7
/proc/30008/stat : 30008 (march_hi) R 30006 30008 30008 0 -1 4202496 1299 0 0 0 17058 0 0 0 25 0 1 0 747539058 6258688 1036 33554432000 4194304 4955518 140736523250928 18446744073709551615 4261398 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/30008/statm: 1528 1036 104 186 0 1341 0
[pid=29969/tid=29976] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/29976/stat : 29976 (python) S 29967 29969 28826 0 -1 4202560 26 934 0 0 0 0 0 0 19 0 11 0 747527203 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 0 0 0 0
[pid=29969/tid=29979] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/29979/stat : 29979 (python) S 29967 29969 28826 0 -1 4202560 23 934 0 0 0 0 0 0 20 0 11 0 747527206 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 3 0 0 0
[pid=29969/tid=29982] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/29982/stat : 29982 (python) S 29967 29969 28826 0 -1 4202560 75 934 0 0 0 0 0 0 19 0 11 0 747527209 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 0 0 0 0
[pid=29969/tid=29985] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/29985/stat : 29985 (python) S 29967 29969 28826 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 11 0 747527212 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 7 0 0 0
[pid=29969/tid=29988] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/29988/stat : 29988 (python) S 29967 29969 28826 0 -1 4202560 35 934 0 0 0 0 0 0 21 0 11 0 747527215 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 1 0 0 0
[pid=29969/tid=29991] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/29991/stat : 29991 (python) S 29967 29969 28826 0 -1 4202560 30 934 0 0 0 0 0 0 22 0 11 0 747527218 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 1 0 0 0
[pid=29969/tid=29994] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/29994/stat : 29994 (python) S 29967 29969 28826 0 -1 4202560 30 934 0 0 0 0 0 0 24 0 11 0 747527223 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 1 0 0 0
[pid=29969/tid=29997] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/29997/stat : 29997 (python) S 29967 29969 28826 0 -1 4202560 44 934 0 0 0 0 0 0 18 0 11 0 747527226 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 1 0 0 0
[pid=29969/tid=30003] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/30003/stat : 30003 (python) S 29967 29969 28826 0 -1 4202560 42 934 0 0 0 0 0 0 19 0 11 0 747537243 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 2 0 0 0
[pid=29969/tid=30007] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/30007/stat : 30007 (python) S 29967 29969 28826 0 -1 4202560 34 934 0 0 0 0 0 0 20 0 11 0 747539058 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2150.58
Current children cumulated vsize (KiB) 6330600

[startup+332.002 s]
/proc/loadavg: 10.27 9.62 9.74 13/511 30015
/proc/meminfo: memFree=16273372/32951124 swapFree=45193380/67111528
[pid=29969] ppid=29967 vsize=447160 CPUtime=0.83 cores=0-7
/proc/29969/stat : 29969 (python) S 29967 29969 28826 0 -1 4202496 17660 934 1 0 71 12 0 0 18 0 11 0 747527147 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296756031 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/29969/statm: 111790 7585 1894 364 0 47681 0
[pid=29975] ppid=29969 vsize=0 CPUtime=100.72 cores=0-7
/proc/29975/stat : 29975 (python) R 29969 29969 28826 0 -1 4202564 1103 437 0 0 18 19 10035 0 18 0 1 0 747527203 0 0 33554432000 0 0 0 0 0 0 0 16781824 2 18446744073709551615 0 0 17 1 0 0 0
/proc/29975/statm: 0 0 0 0 0 0 0
[pid=29978] ppid=29969 vsize=289332 CPUtime=1.39 cores=0-7
/proc/29978/stat : 29978 (python) S 29969 29969 28826 0 -1 4202560 998 0 0 0 75 64 0 0 15 0 1 0 747527206 296275968 5713 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 4579896942540062979 0 0 17 3 0 0 0
/proc/29978/statm: 72333 5713 371 364 0 8248 0
[pid=29980] ppid=29978 vsize=11060 CPUtime=297.84 cores=0-7
/proc/29980/stat : 29980 (march_hi) R 29978 29980 29980 0 -1 4202496 2570 0 0 0 29783 1 0 0 25 0 1 0 747527206 11325440 2305 33554432000 4194304 4955518 140733355312896 18446744073709551615 4227756 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/29980/statm: 2765 2305 104 186 0 2578 0
[pid=29981] ppid=29969 vsize=0 CPUtime=308.21 cores=0-7
/proc/29981/stat : 29981 (python) R 29969 29969 28826 0 -1 4202564 1101 2678 0 0 263 89 30466 3 19 0 1 0 747527209 0 0 33554432000 0 0 0 0 0 0 0 16781824 2 18446744073709551615 0 0 17 0 0 0 0
/proc/29981/statm: 0 0 0 0 0 0 0
[pid=29984] ppid=29969 vsize=375380 CPUtime=0.43 cores=0-7
/proc/29984/stat : 29984 (python) S 29969 29969 28826 0 -1 4202560 970 0 0 0 21 22 0 0 19 0 2 0 747527212 384389120 5881 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296752587 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/29984/statm: 93845 5881 400 364 0 29754 0
[pid=29986] ppid=29984 vsize=1017800 CPUtime=100.63 cores=0-7
/proc/29986/stat : 29986 (gnovelty+2) T 29984 29986 29986 0 -1 4202496 430 0 0 0 10063 0 0 0 25 0 1 0 747527212 1042227200 174 33554432000 134512640 135663704 4290844560 18446744073709551615 134525684 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/29986/statm: 254450 174 113 282 0 254166 0
[pid=29984/tid=30005] ppid=29969 vsize=375380 CPUtime=0 cores=0-7
/proc/29984/task/30005/stat : 30005 (python) S 29969 29969 28826 0 -1 4202560 67 0 0 0 0 0 0 0 19 0 2 0 747539055 384389120 5881 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 514 18446604440321913152 0 0 -1 0 0 0 0
[pid=29987] ppid=29969 vsize=385636 CPUtime=1.35 cores=0-7
/proc/29987/stat : 29987 (python) S 29969 29969 28826 0 -1 4202560 817 0 0 0 69 66 0 0 15 0 1 0 747527215 394891264 5873 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 7505187210656710914 0 0 17 2 0 0 0
/proc/29987/statm: 96409 5873 371 364 0 32315 0
[pid=29989] ppid=29987 vsize=11060 CPUtime=263.98 cores=0-7
/proc/29989/stat : 29989 (march_hi) R 29987 29989 29989 0 -1 4202496 2114 0 0 0 26397 1 0 0 25 0 1 0 747527217 11325440 1861 33554432000 4194304 4955518 140733277752256 18446744073709551615 4209717 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/29989/statm: 2765 1861 104 186 0 2578 0
[pid=29990] ppid=29969 vsize=395892 CPUtime=1.34 cores=0-7
/proc/29990/stat : 29990 (python) S 29969 29969 28826 0 -1 4202560 823 0 0 0 72 62 0 0 15 0 1 0 747527218 405393408 5895 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 7356502452255817985 0 0 17 6 0 0 0
/proc/29990/statm: 98973 5895 372 364 0 34876 0
[pid=29992] ppid=29990 vsize=11060 CPUtime=265.71 cores=0-7
/proc/29992/stat : 29992 (march_hi) R 29990 29992 29992 0 -1 4202496 2143 0 0 0 26571 0 0 0 25 0 1 0 747527222 11325440 1884 33554432000 4194304 4955518 140735650732992 18446744073709551615 4227970 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/29992/statm: 2765 1884 104 186 0 2578 0
[pid=29993] ppid=29969 vsize=406148 CPUtime=1.35 cores=0-7
/proc/29993/stat : 29993 (python) S 29969 29969 28826 0 -1 4202560 833 0 0 0 67 68 0 0 15 0 1 0 747527223 415895552 5898 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 412796251780645124 0 0 17 1 0 0 0
/proc/29993/statm: 101537 5898 371 364 0 37437 0
[pid=29995] ppid=29993 vsize=8048 CPUtime=213.11 cores=0-7
/proc/29995/stat : 29995 (march_hi) R 29993 29995 29995 0 -1 4202496 1617 0 0 0 21310 1 0 0 25 0 1 0 747527225 8241152 1356 33554432000 4194304 4955518 140734727806448 18446744073709551615 4224642 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/29995/statm: 2012 1356 104 186 0 1825 0
[pid=29996] ppid=29969 vsize=416404 CPUtime=1.34 cores=0-7
/proc/29996/stat : 29996 (python) S 29969 29969 28826 0 -1 4202560 1065 0 0 0 70 64 0 0 15 0 2 0 747527226 426397696 5955 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/29996/statm: 104101 5955 406 364 0 39998 0
[pid=29998] ppid=29996 vsize=8048 CPUtime=238.41 cores=0-7
/proc/29998/stat : 29998 (march_hi) R 29996 29998 29998 0 -1 4202496 1860 0 0 0 23840 1 0 0 25 0 1 0 747527226 8241152 1594 33554432000 4194304 4955518 140735913765408 18446744073709551615 4227565 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/29998/statm: 2012 1594 104 186 0 1825 0
[pid=29996/tid=30009] ppid=29969 vsize=416404 CPUtime=0 cores=0-7
/proc/29996/task/30009/stat : 30009 (python) S 29969 29969 28826 0 -1 4202560 83 0 0 0 0 0 0 0 15 0 2 0 747540807 426397696 5955 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 514 18446604457272032256 0 0 -1 0 0 0 0
[pid=30002] ppid=29969 vsize=426660 CPUtime=0.93 cores=0-7
/proc/30002/stat : 30002 (python) S 29969 29969 28826 0 -1 4202560 850 0 0 0 50 43 0 0 15 0 1 0 747537243 436899840 5928 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 3960634401590575367 0 0 17 4 0 0 0
/proc/30002/statm: 106665 5928 371 364 0 42559 0
[pid=30004] ppid=30002 vsize=6112 CPUtime=182.2 cores=0-7
/proc/30004/stat : 30004 (march_hi) R 30002 30004 30004 0 -1 4202496 1385 0 0 0 18220 0 0 0 25 0 1 0 747537243 6258688 1119 33554432000 4194304 4955518 140733951866352 18446744073709551615 4286525 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/30004/statm: 1528 1119 104 186 0 1341 0
[pid=30006] ppid=29969 vsize=436916 CPUtime=0.87 cores=0-7
/proc/30006/stat : 30006 (python) S 29969 29969 28826 0 -1 4202560 866 0 0 0 44 43 0 0 16 0 1 0 747539058 447401984 5975 33554432000 4194304 5683644 140733259638400 18446744073709551615 224284945990 0 0 16781312 514 18216902163039224068 0 0 17 1 0 0 0
/proc/30006/statm: 109229 5975 371 364 0 45120 0
[pid=30008] ppid=30006 vsize=6112 CPUtime=170.64 cores=0-7
/proc/30008/stat : 30008 (march_hi) R 30006 30008 30008 0 -1 4202496 1299 0 0 0 17064 0 0 0 25 0 1 0 747539058 6258688 1036 33554432000 4194304 4955518 140736523250928 18446744073709551615 4209717 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/30008/statm: 1528 1036 104 186 0 1341 0
[pid=29969/tid=29976] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/29976/stat : 29976 (python) S 29967 29969 28826 0 -1 4202560 26 934 0 0 0 0 0 0 19 0 11 0 747527203 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 0 0 0 0
[pid=29969/tid=29979] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/29979/stat : 29979 (python) S 29967 29969 28826 0 -1 4202560 23 934 0 0 0 0 0 0 20 0 11 0 747527206 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 3 0 0 0
[pid=29969/tid=29982] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/29982/stat : 29982 (python) S 29967 29969 28826 0 -1 4202560 75 934 0 0 0 0 0 0 19 0 11 0 747527209 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 0 0 0 0
[pid=29969/tid=29985] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/29985/stat : 29985 (python) S 29967 29969 28826 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 11 0 747527212 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 7 0 0 0
[pid=29969/tid=29988] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/29988/stat : 29988 (python) S 29967 29969 28826 0 -1 4202560 35 934 0 0 0 0 0 0 21 0 11 0 747527215 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 1 0 0 0
[pid=29969/tid=29991] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/29991/stat : 29991 (python) S 29967 29969 28826 0 -1 4202560 30 934 0 0 0 0 0 0 22 0 11 0 747527218 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 1 0 0 0
[pid=29969/tid=29994] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/29994/stat : 29994 (python) S 29967 29969 28826 0 -1 4202560 30 934 0 0 0 0 0 0 24 0 11 0 747527223 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 1 0 0 0
[pid=29969/tid=29997] ppid=29967 vsize=447160 CPUtime=0 cores=0-7
/proc/29969/task/29997/stat : 29997 (python) S 29967 29969 28826 0 -1 4202560 44 934 0 0 0 0 0 0 18 0 11 0 747527226 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 1 0 0 0
[pid=29969/tid=30003] ppid=29967 vsize=447160 CPUtime=100.72 cores=0-7
/proc/29969/task/30003/stat : 30003 (python) S 29967 29969 28826 0 -1 4202560 42 2474 0 0 0 0 10053 19 19 0 11 0 747537243 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 2 0 0 0
[pid=29969/tid=30007] ppid=29967 vsize=447160 CPUtime=100.72 cores=0-7
/proc/29969/task/30007/stat : 30007 (python) S 29967 29969 28826 0 -1 4202560 34 2474 0 0 0 0 10053 19 20 0 11 0 747539058 457891840 7585 33554432000 4194304 5683644 140733259638400 18446744073709551615 224296749489 0 0 16781312 2 18446604450163525504 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2151.28
Current children cumulated vsize (KiB) 4658828

Child status: 0
Real time (s): 332.037
CPU time (s): 2151.56
CPU user time (s): 2145.86
CPU system time (s): 5.69713
CPU usage (%): 647.987
Max. virtual memory (cumulated for all children) (KiB): 6338200

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2145.86
system time used= 5.69713
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 49044
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 6993
involuntary context switches= 18005

runsolver used 2.40863 second user time and 5.14022 second system time

The end

Launcher Data 

Begin job on node118 at 2011-05-16 21:09:25
IDJOB=3385751
IDBENCH=83016
IDSOLVER=1870
FILE ID=node118/3385751-1305572965
RUNJOBID= node118-1305564964-28840
PBS_JOBID= 13350132
Free space on /tmp= 73156 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v450-c1917-S1879031150-062.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385751-1305572965/watcher-3385751-1305572965 -o /tmp/evaluation-result-3385751-1305572965/solver-3385751-1305572965 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385751-1305572965.cnf -seed 511261501 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= dea41a8eae52b60f8e05be01aec1734d
RANDOM SEED=511261501

node118.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5374.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      16376904 kB
Buffers:        319184 kB
Cached:        4954260 kB
SwapCached:      22068 kB
Active:       13499564 kB
Inactive:      2737848 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      16376904 kB
SwapTotal:    67111528 kB
SwapFree:     45193380 kB
Dirty:           32292 kB
Writeback:           0 kB
AnonPages:    10955524 kB
Mapped:          25112 kB
Slab:           160356 kB
PageTables:     105124 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 151000960 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73124 MiB
End job on node118 at 2011-05-16 21:14:57