Trace number 3322015

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-randomUNSAT 2578.67 323.599

General information on the benchmark

Namerandom/medium/
unif-k5-r21.3-v100-c2130-S1760258375-046.UNKNOWN.cnf
MD5SUMf913d39a64a2142ed34486c38d5636c0
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark320.093
Satisfiable
(Un)Satisfiability was proved
Number of variables100
Number of clauses2130
Sum of the clauses size10650
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 52130
Number of clauses of size over 50

Solver Data 

0.32/0.47	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.32/0.47	c INFO: solving HOME/instance-3322015-1304370801.cnf
0.32/0.47	c INFO: computing task features
0.32/0.48	c INFO: computed variable-clause graph statistics
0.32/0.48	c INFO: computed clause balance statistics
0.40/0.54	c INFO: computed variable balance statistics
0.40/0.57	c INFO: computed small-clause counts
0.40/0.57	c INFO: computed Horn-clause counts
0.40/0.57	c INFO: computed variable graph statistics
0.40/0.57	c INFO: collected features for HOME/instance-3322015-1304370801.cnf in 0.10 s
0.50/0.61	c INFO: running gnovelty+2@0 for 97 with 39999 remaining (b = 0.62)
0.50/0.64	c INFO: running march_hi@0 for 194 with 39999 remaining (b = 0.18)
0.50/0.68	c INFO: running gnovelty+2@0 for 97 with 39999 remaining (b = 0.07)
0.60/0.72	c INFO: running march_hi@0 for 388 with 39999 remaining (b = 0.29)
0.60/0.76	c INFO: running march_hi@0 for 388 with 39999 remaining (b = 0.25)
0.70/0.80	c INFO: running march_hi@0 for 485 with 39999 remaining (b = 0.27)
0.70/0.84	c INFO: running march_hi@0 for 485 with 39999 remaining (b = 0.24)
0.70/0.88	c INFO: running march_hi@0 for 388 with 39999 remaining (b = 0.17)
800.19/100.83	c INFO: running kcnfs-2006@0 for 582 with 39899 remaining (b = 0.20)
800.92/100.91	c INFO: running march_hi@0 for 388 with 39899 remaining (b = 0.13)
1566.55/196.86	c INFO: running march_hi@0 for 97 with 39803 remaining (b = 0.04)
2339.26/293.61	c INFO: running march_hi@96 for 97 with 39706 remaining (b = 0.04)
2577.87/323.57	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3322015-1304370801/watcher-3322015-1304370801 -o /tmp/evaluation-result-3322015-1304370801/solver-3322015-1304370801 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3322015-1304370801.cnf -seed 1031125028 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.05 8.50 8.29 1/266 31505
/proc/meminfo: memFree=28107208/32951124 swapFree=67107192/67111528
[pid=31505] ppid=31503 vsize=15776 CPUtime=0 cores=0-7
/proc/31505/stat : 31505 (python) S 31503 31505 30480 0 -1 4202496 896 0 0 0 0 0 0 0 25 0 1 0 548678624 16154624 777 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204645376 0 2147483391 16781312 2 18446744071699207358 0 0 17 6 0 0 0
/proc/31505/statm: 3944 777 343 364 0 431 0

[startup+0.062079 s]
/proc/loadavg: 8.05 8.50 8.29 1/266 31505
/proc/meminfo: memFree=28107208/32951124 swapFree=67107192/67111528
[pid=31505] ppid=31503 vsize=40972 CPUtime=0.04 cores=0-7
/proc/31505/stat : 31505 (python) S 31503 31505 30480 0 -1 4202496 1678 0 0 0 3 1 0 0 25 0 1 0 548678624 41955328 1502 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204645376 0 2147483391 16781312 2 18446744071699207358 0 0 17 6 0 0 0
/proc/31505/statm: 10243 1502 463 364 0 1026 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 40972

[startup+0.101073 s]
/proc/loadavg: 8.05 8.50 8.29 1/266 31505
/proc/meminfo: memFree=28107208/32951124 swapFree=67107192/67111528
[pid=31505] ppid=31503 vsize=104224 CPUtime=0.07 cores=0-7
/proc/31505/stat : 31505 (python) R 31503 31505 30480 0 -1 4202496 3059 0 0 0 5 2 0 0 25 0 1 0 548678624 106725376 2741 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204645376 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/31505/statm: 26056 2741 847 364 0 1804 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 104224

[startup+0.301048 s]
/proc/loadavg: 8.05 8.50 8.29 1/266 31505
/proc/meminfo: memFree=28107208/32951124 swapFree=67107192/67111528
[pid=31505] ppid=31503 vsize=170752 CPUtime=0.23 cores=0-7
/proc/31505/stat : 31505 (python) R 31503 31505 30480 0 -1 4202496 6735 934 1 0 17 6 0 0 17 0 1 0 548678624 174850048 4806 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204643173 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/31505/statm: 42688 4806 1336 364 0 3453 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 170752

[startup+0.700986 s]
/proc/loadavg: 8.05 8.50 8.29 1/266 31505
/proc/meminfo: memFree=28107208/32951124 swapFree=67107192/67111528
[pid=31505] ppid=31503 vsize=375316 CPUtime=0.6 cores=0-7
/proc/31505/stat : 31505 (python) R 31503 31505 30480 0 -1 4202496 11769 934 1 0 51 9 0 0 18 0 4 0 548678624 384323584 7328 33554432000 4194304 5683644 140734210798320 18446744073709551615 210208085682 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31505/statm: 93829 7328 1878 364 0 29741 0
Current children cumulated CPU time (s) 0.6
Current children cumulated vsize (KiB) 375316

[startup+1.50179 s]
/proc/loadavg: 7.49 8.38 8.25 10/291 31534
/proc/meminfo: memFree=28052688/32951124 swapFree=67107192/67111528
[pid=31505] ppid=31503 vsize=426596 CPUtime=0.78 cores=0-7
/proc/31505/stat : 31505 (python) S 31503 31505 30480 0 -1 4202496 15303 934 1 0 68 10 0 0 25 0 9 0 548678624 436834304 7408 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216474059 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/31505/statm: 106649 7408 1889 364 0 42546 0
[pid=31511] ppid=31505 vsize=278588 CPUtime=0 cores=0-7
/proc/31511/stat : 31511 (python) S 31505 31505 30480 0 -1 4202560 737 0 0 0 0 0 0 0 18 0 1 0 548678685 285274112 5560 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667407 0 0 16781312 514 10389546234284081151 0 0 17 1 0 0 0
/proc/31511/statm: 69647 5560 354 364 0 5565 0
[pid=31513] ppid=31511 vsize=1017800 CPUtime=0.88 cores=0-7
/proc/31513/stat : 31513 (gnovelty+2) R 31511 31513 31513 0 -1 4202496 437 0 0 0 88 0 0 0 21 0 1 0 548678685 1042227200 172 33554432000 134512640 135663704 4288779728 18446744073709551615 134525691 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31513/statm: 254450 172 113 282 0 254166 0
[pid=31514] ppid=31505 vsize=289280 CPUtime=0 cores=0-7
/proc/31514/stat : 31514 (python) S 31505 31505 30480 0 -1 4202560 746 0 0 0 0 0 0 0 15 0 1 0 548678689 296222720 5677 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 17739978003670073603 0 0 17 1 0 0 0
/proc/31514/statm: 72320 5677 348 364 0 8235 0
[pid=31516] ppid=31514 vsize=1560 CPUtime=0.84 cores=0-7
/proc/31516/stat : 31516 (march_hi) R 31514 31516 31516 0 -1 4202496 501 0 0 0 84 0 0 0 18 0 1 0 548678689 1597440 240 33554432000 4194304 4955518 140736054912704 18446744073709551615 4297472 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31516/statm: 390 240 106 186 0 203 0
[pid=31517] ppid=31505 vsize=365072 CPUtime=0 cores=0-7
/proc/31517/stat : 31517 (python) S 31505 31505 30480 0 -1 4202560 710 0 0 0 0 0 0 0 19 0 1 0 548678692 373833728 5854 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 15239956568066752511 0 0 17 1 0 0 0
/proc/31517/statm: 91268 5854 348 364 0 27180 0
[pid=31519] ppid=31517 vsize=1017800 CPUtime=0.8 cores=0-7
/proc/31519/stat : 31519 (gnovelty+2) R 31517 31519 31519 0 -1 4202496 423 0 0 0 80 0 0 0 22 0 1 0 548678693 1042227200 171 33554432000 134512640 135663704 4292807776 18446744073709551615 134524888 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/31519/statm: 254450 171 113 282 0 254166 0
[pid=31520] ppid=31505 vsize=375328 CPUtime=0 cores=0-7
/proc/31520/stat : 31520 (python) S 31505 31505 30480 0 -1 4202560 720 0 0 0 0 0 0 0 15 0 1 0 548678696 384335872 5830 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 16168331684880548098 0 0 17 6 0 0 0
/proc/31520/statm: 93832 5830 348 364 0 29741 0
[pid=31522] ppid=31520 vsize=1560 CPUtime=0.76 cores=0-7
/proc/31522/stat : 31522 (march_hi) R 31520 31522 31522 0 -1 4202496 493 0 0 0 76 0 0 0 22 0 1 0 548678696 1597440 240 33554432000 4194304 4955518 140736994141952 18446744073709551615 4314278 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31522/statm: 390 240 106 186 0 203 0
[pid=31523] ppid=31505 vsize=385584 CPUtime=0 cores=0-7
/proc/31523/stat : 31523 (python) S 31505 31505 30480 0 -1 4202560 731 0 0 0 0 0 0 0 15 0 1 0 548678700 394838016 5851 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 7215461498691223810 0 0 17 7 0 0 0
/proc/31523/statm: 96396 5851 348 364 0 32302 0
[pid=31525] ppid=31523 vsize=1560 CPUtime=0.72 cores=0-7
/proc/31525/stat : 31525 (march_hi) R 31523 31525 31525 0 -1 4202496 496 0 0 0 72 0 0 0 25 0 1 0 548678700 1597440 240 33554432000 4194304 4955518 140735285938032 18446744073709551615 4235624 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31525/statm: 390 240 106 186 0 203 0
[pid=31526] ppid=31505 vsize=395840 CPUtime=0 cores=0-7
/proc/31526/stat : 31526 (python) S 31505 31505 30480 0 -1 4202560 738 0 0 0 0 0 0 0 16 0 1 0 548678704 405340160 5821 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 10849356224583926020 0 0 17 1 0 0 0
/proc/31526/statm: 98960 5821 348 364 0 34863 0
[pid=31528] ppid=31526 vsize=1560 CPUtime=0.69 cores=0-7
/proc/31528/stat : 31528 (march_hi) R 31526 31528 31528 0 -1 4202496 496 0 0 0 69 0 0 0 25 0 1 0 548678704 1597440 239 33554432000 4194304 4955518 140734064562608 18446744073709551615 4208638 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31528/statm: 390 239 106 186 0 203 0
[pid=31529] ppid=31505 vsize=406096 CPUtime=0 cores=0-7
/proc/31529/stat : 31529 (python) S 31505 31505 30480 0 -1 4202560 748 0 0 0 0 0 0 0 19 0 1 0 548678708 415842304 5945 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 9465660423446429956 0 0 17 0 0 0 0
/proc/31529/statm: 101524 5945 348 364 0 37424 0
[pid=31531] ppid=31529 vsize=1560 CPUtime=0.65 cores=0-7
/proc/31531/stat : 31531 (march_hi) R 31529 31531 31531 0 -1 4202496 499 0 0 0 65 0 0 0 25 0 1 0 548678708 1597440 240 33554432000 4194304 4955518 140737080243728 18446744073709551615 4208667 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31531/statm: 390 240 106 186 0 203 0
[pid=31532] ppid=31505 vsize=416352 CPUtime=0 cores=0-7
/proc/31532/stat : 31532 (python) S 31505 31505 30480 0 -1 4202560 781 0 0 0 0 0 0 0 21 0 1 0 548678712 426344448 5869 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 5602229945037193473 0 0 17 4 0 0 0
/proc/31532/statm: 104088 5869 354 364 0 39985 0
[pid=31534] ppid=31532 vsize=1560 CPUtime=0.6 cores=0-7
/proc/31534/stat : 31534 (march_hi) R 31532 31534 31534 0 -1 4202496 504 0 0 0 60 0 0 0 25 0 1 0 548678712 1597440 240 33554432000 4194304 4955518 140736787511600 18446744073709551615 4224450 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31534/statm: 390 240 106 186 0 203 0
[pid=31505/tid=31512] ppid=31503 vsize=426596 CPUtime=0 cores=0-7
/proc/31505/task/31512/stat : 31512 (python) S 31503 31505 30480 0 -1 4202560 29 934 0 0 0 0 0 0 18 0 9 0 548678685 436834304 7408 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 4 0 0 0
[pid=31505/tid=31515] ppid=31503 vsize=426596 CPUtime=0 cores=0-7
/proc/31505/task/31515/stat : 31515 (python) S 31503 31505 30480 0 -1 4202560 26 934 0 0 0 0 0 0 18 0 9 0 548678689 436834304 7408 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 3 0 0 0
[pid=31505/tid=31518] ppid=31503 vsize=426596 CPUtime=0 cores=0-7
/proc/31505/task/31518/stat : 31518 (python) S 31503 31505 30480 0 -1 4202560 81 934 0 0 0 0 0 0 19 0 9 0 548678693 436834304 7408 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 0 0 0 0
[pid=31505/tid=31521] ppid=31503 vsize=426596 CPUtime=0 cores=0-7
/proc/31505/task/31521/stat : 31521 (python) S 31503 31505 30480 0 -1 4202560 32 934 0 0 0 0 0 0 20 0 9 0 548678696 436834304 7408 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 0 0 0 0
[pid=31505/tid=31524] ppid=31503 vsize=426596 CPUtime=0 cores=0-7
/proc/31505/task/31524/stat : 31524 (python) S 31503 31505 30480 0 -1 4202560 33 934 0 0 0 0 0 0 21 0 9 0 548678700 436834304 7408 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 0 0 0 0
[pid=31505/tid=31527] ppid=31503 vsize=426596 CPUtime=0 cores=0-7
/proc/31505/task/31527/stat : 31527 (python) S 31503 31505 30480 0 -1 4202560 31 934 0 0 0 0 0 0 23 0 9 0 548678704 436834304 7408 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 4 0 0 0
[pid=31505/tid=31530] ppid=31503 vsize=426596 CPUtime=0 cores=0-7
/proc/31505/task/31530/stat : 31530 (python) S 31503 31505 30480 0 -1 4202560 33 934 0 0 0 0 0 0 25 0 9 0 548678708 436834304 7408 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 4 0 0 0
[pid=31505/tid=31533] ppid=31503 vsize=426596 CPUtime=0 cores=0-7
/proc/31505/task/31533/stat : 31533 (python) S 31503 31505 30480 0 -1 4202560 34 934 0 0 0 0 0 0 25 0 9 0 548678712 436834304 7408 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 6.72
Current children cumulated vsize (KiB) 5383696

[startup+3.10152 s]
/proc/loadavg: 7.49 8.38 8.25 9/291 31534
/proc/meminfo: memFree=28052812/32951124 swapFree=67107192/67111528
[pid=31505] ppid=31503 vsize=426596 CPUtime=0.78 cores=0-7
/proc/31505/stat : 31505 (python) S 31503 31505 30480 0 -1 4202496 15303 934 1 0 68 10 0 0 25 0 9 0 548678624 436834304 7408 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216474059 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/31505/statm: 106649 7408 1889 364 0 42546 0
[pid=31511] ppid=31505 vsize=278588 CPUtime=0 cores=0-7
/proc/31511/stat : 31511 (python) S 31505 31505 30480 0 -1 4202560 737 0 0 0 0 0 0 0 18 0 1 0 548678685 285274112 5560 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667407 0 0 16781312 514 10389546234284048643 0 0 17 1 0 0 0
/proc/31511/statm: 69647 5560 354 364 0 5565 0
[pid=31513] ppid=31511 vsize=1017800 CPUtime=2.48 cores=0-7
/proc/31513/stat : 31513 (gnovelty+2) R 31511 31513 31513 0 -1 4202496 437 0 0 0 248 0 0 0 25 0 1 0 548678685 1042227200 172 33554432000 134512640 135663704 4288779728 18446744073709551615 134525265 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31513/statm: 254450 172 113 282 0 254166 0
[pid=31514] ppid=31505 vsize=289280 CPUtime=0 cores=0-7
/proc/31514/stat : 31514 (python) S 31505 31505 30480 0 -1 4202560 749 0 0 0 0 0 0 0 15 0 1 0 548678689 296222720 5677 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 17739978003670073603 0 0 17 1 0 0 0
/proc/31514/statm: 72320 5677 348 364 0 8235 0
[pid=31516] ppid=31514 vsize=1560 CPUtime=2.44 cores=0-7
/proc/31516/stat : 31516 (march_hi) R 31514 31516 31516 0 -1 4202496 504 0 0 0 244 0 0 0 22 0 1 0 548678689 1597440 243 33554432000 4194304 4955518 140736054912704 18446744073709551615 4208667 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31516/statm: 390 243 106 186 0 203 0
[pid=31517] ppid=31505 vsize=365072 CPUtime=0 cores=0-7
/proc/31517/stat : 31517 (python) S 31505 31505 30480 0 -1 4202560 710 0 0 0 0 0 0 0 18 0 1 0 548678692 373833728 5854 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 15239956568066720002 0 0 17 1 0 0 0
/proc/31517/statm: 91268 5854 348 364 0 27180 0
[pid=31519] ppid=31517 vsize=1017800 CPUtime=2.4 cores=0-7
/proc/31519/stat : 31519 (gnovelty+2) R 31517 31519 31519 0 -1 4202496 423 0 0 0 240 0 0 0 25 0 1 0 548678693 1042227200 171 33554432000 134512640 135663704 4292807776 18446744073709551615 134525255 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/31519/statm: 254450 171 113 282 0 254166 0
[pid=31520] ppid=31505 vsize=375328 CPUtime=0 cores=0-7
/proc/31520/stat : 31520 (python) S 31505 31505 30480 0 -1 4202560 723 0 0 0 0 0 0 0 15 0 1 0 548678696 384335872 5830 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 16168331684880548098 0 0 17 6 0 0 0
/proc/31520/statm: 93832 5830 348 364 0 29741 0
[pid=31522] ppid=31520 vsize=1560 CPUtime=2.36 cores=0-7
/proc/31522/stat : 31522 (march_hi) R 31520 31522 31522 0 -1 4202496 496 0 0 0 236 0 0 0 25 0 1 0 548678696 1597440 243 33554432000 4194304 4955518 140736994141952 18446744073709551615 4235617 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31522/statm: 390 243 106 186 0 203 0
[pid=31523] ppid=31505 vsize=385584 CPUtime=0 cores=0-7
/proc/31523/stat : 31523 (python) S 31505 31505 30480 0 -1 4202560 734 0 0 0 0 0 0 0 15 0 1 0 548678700 394838016 5851 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 7215461498691223810 0 0 17 7 0 0 0
/proc/31523/statm: 96396 5851 348 364 0 32302 0
[pid=31525] ppid=31523 vsize=1560 CPUtime=2.31 cores=0-7
/proc/31525/stat : 31525 (march_hi) R 31523 31525 31525 0 -1 4202496 498 0 0 0 231 0 0 0 25 0 1 0 548678700 1597440 242 33554432000 4194304 4955518 140735285938032 18446744073709551615 4208667 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31525/statm: 390 242 106 186 0 203 0
[pid=31526] ppid=31505 vsize=395840 CPUtime=0 cores=0-7
/proc/31526/stat : 31526 (python) S 31505 31505 30480 0 -1 4202560 741 0 0 0 0 0 0 0 15 0 1 0 548678704 405340160 5821 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 10849356224583926020 0 0 17 1 0 0 0
/proc/31526/statm: 98960 5821 348 364 0 34863 0
[pid=31528] ppid=31526 vsize=1560 CPUtime=2.28 cores=0-7
/proc/31528/stat : 31528 (march_hi) R 31526 31528 31528 0 -1 4202496 498 0 0 0 228 0 0 0 25 0 1 0 548678704 1597440 241 33554432000 4194304 4955518 140734064562608 18446744073709551615 4208667 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31528/statm: 390 241 106 186 0 203 0
[pid=31529] ppid=31505 vsize=406096 CPUtime=0 cores=0-7
/proc/31529/stat : 31529 (python) S 31505 31505 30480 0 -1 4202560 748 0 0 0 0 0 0 0 15 0 1 0 548678708 415842304 5945 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 9465660423446429956 0 0 17 0 0 0 0
/proc/31529/statm: 101524 5945 348 364 0 37424 0
[pid=31531] ppid=31529 vsize=1560 CPUtime=2.25 cores=0-7
/proc/31531/stat : 31531 (march_hi) R 31529 31531 31531 0 -1 4202496 501 0 0 0 225 0 0 0 25 0 1 0 548678708 1597440 242 33554432000 4194304 4955518 140737080243728 18446744073709551615 4237810 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31531/statm: 390 242 106 186 0 203 0
[pid=31532] ppid=31505 vsize=416352 CPUtime=0 cores=0-7
/proc/31532/stat : 31532 (python) S 31505 31505 30480 0 -1 4202560 786 0 0 0 0 0 0 0 15 0 1 0 548678712 426344448 5869 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 5602229949332160769 0 0 17 4 0 0 0
/proc/31532/statm: 104088 5869 354 364 0 39985 0
[pid=31534] ppid=31532 vsize=1560 CPUtime=2.2 cores=0-7
/proc/31534/stat : 31534 (march_hi) R 31532 31534 31534 0 -1 4202496 506 0 0 0 220 0 0 0 25 0 1 0 548678712 1597440 242 33554432000 4194304 4955518 140736787511600 18446744073709551615 4235379 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31534/statm: 390 242 106 186 0 203 0
[pid=31505/tid=31512] ppid=31503 vsize=426596 CPUtime=0 cores=0-7
/proc/31505/task/31512/stat : 31512 (python) S 31503 31505 30480 0 -1 4202560 29 934 0 0 0 0 0 0 18 0 9 0 548678685 436834304 7408 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 4 0 0 0
[pid=31505/tid=31515] ppid=31503 vsize=426596 CPUtime=0 cores=0-7
/proc/31505/task/31515/stat : 31515 (python) S 31503 31505 30480 0 -1 4202560 26 934 0 0 0 0 0 0 18 0 9 0 548678689 436834304 7408 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 3 0 0 0
[pid=31505/tid=31518] ppid=31503 vsize=426596 CPUtime=0 cores=0-7
/proc/31505/task/31518/stat : 31518 (python) S 31503 31505 30480 0 -1 4202560 81 934 0 0 0 0 0 0 19 0 9 0 548678693 436834304 7408 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 0 0 0 0
[pid=31505/tid=31521] ppid=31503 vsize=426596 CPUtime=0 cores=0-7
/proc/31505/task/31521/stat : 31521 (python) S 31503 31505 30480 0 -1 4202560 32 934 0 0 0 0 0 0 20 0 9 0 548678696 436834304 7408 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 0 0 0 0
[pid=31505/tid=31524] ppid=31503 vsize=426596 CPUtime=0 cores=0-7

################
# More data... #
################

/proc/31516/statm: 3378 2837 106 186 0 3191 0
[pid=31514/tid=31549] ppid=31505 vsize=289280 CPUtime=0 cores=0-7
/proc/31514/task/31549/stat : 31549 (python) S 31505 31505 30480 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 548698305 296222720 5730 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 514 18446604452936784640 0 0 -1 1 0 0 0
[pid=31517] ppid=31505 vsize=365072 CPUtime=0.18 cores=0-7
/proc/31517/stat : 31517 (python) S 31505 31505 30480 0 -1 4202560 1006 0 0 0 9 9 0 0 18 0 2 0 548678692 373833728 5907 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216474059 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/31517/statm: 91268 5907 400 364 0 27180 0
[pid=31519] ppid=31517 vsize=1017800 CPUtime=100.16 cores=0-7
/proc/31519/stat : 31519 (gnovelty+2) T 31517 31519 31519 0 -1 4202496 423 0 0 0 10016 0 0 0 25 0 1 0 548678693 1042227200 171 33554432000 134512640 135663704 4292807776 18446744073709551615 134524945 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/31519/statm: 254450 171 113 282 0 254166 0
[pid=31517/tid=31542] ppid=31505 vsize=365072 CPUtime=0 cores=0-7
/proc/31517/task/31542/stat : 31542 (python) S 31505 31505 30480 0 -1 4202560 114 0 0 0 0 0 0 0 19 0 2 0 548688710 373833728 5907 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 514 18446604447871087552 0 0 -1 5 0 0 0
[pid=31520] ppid=31505 vsize=375328 CPUtime=0.72 cores=0-7
/proc/31520/stat : 31520 (python) S 31505 31505 30480 0 -1 4202560 818 0 0 0 39 33 0 0 15 0 1 0 548678696 384335872 5853 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 16168331684880548098 0 0 17 6 0 0 0
/proc/31520/statm: 93832 5853 371 364 0 29741 0
[pid=31522] ppid=31520 vsize=13512 CPUtime=321.64 cores=0-7
/proc/31522/stat : 31522 (march_hi) R 31520 31522 31522 0 -1 4202496 3108 0 0 0 32164 0 0 0 25 0 1 0 548678696 13836288 2837 33554432000 4194304 4955518 140736994141952 18446744073709551615 4296850 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31522/statm: 3378 2837 106 186 0 3191 0
[pid=31523] ppid=31505 vsize=385584 CPUtime=0.72 cores=0-7
/proc/31523/stat : 31523 (python) S 31505 31505 30480 0 -1 4202560 817 0 0 0 39 33 0 0 15 0 1 0 548678700 394838016 5874 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 7215461498691223810 0 0 17 7 0 0 0
/proc/31523/statm: 96396 5874 371 364 0 32302 0
[pid=31525] ppid=31523 vsize=13512 CPUtime=316.93 cores=0-7
/proc/31525/stat : 31525 (march_hi) R 31523 31525 31525 0 -1 4202496 3029 0 0 0 31693 0 0 0 25 0 1 0 548678700 13836288 2755 33554432000 4194304 4955518 140735285938032 18446744073709551615 4208656 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31525/statm: 3378 2755 106 186 0 3191 0
[pid=31526] ppid=31505 vsize=395840 CPUtime=0.71 cores=0-7
/proc/31526/stat : 31526 (python) S 31505 31505 30480 0 -1 4202560 833 0 0 0 39 32 0 0 15 0 1 0 548678704 405340160 5845 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 10849356224583926020 0 0 17 1 0 0 0
/proc/31526/statm: 98960 5845 372 364 0 34863 0
[pid=31528] ppid=31526 vsize=13512 CPUtime=320.47 cores=0-7
/proc/31528/stat : 31528 (march_hi) R 31526 31528 31528 0 -1 4202496 3120 0 0 0 32045 2 0 0 25 0 1 0 548678704 13836288 2845 33554432000 4194304 4955518 140734064562608 18446744073709551615 4297540 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31528/statm: 3378 2845 106 186 0 3191 0
[pid=31529] ppid=31505 vsize=406096 CPUtime=0.73 cores=0-7
/proc/31529/stat : 31529 (python) S 31505 31505 30480 0 -1 4202560 830 0 0 0 41 32 0 0 15 0 1 0 548678708 415842304 5968 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 9465660423446429956 0 0 17 0 0 0 0
/proc/31529/statm: 101524 5968 371 364 0 37424 0
[pid=31531] ppid=31529 vsize=13512 CPUtime=321.52 cores=0-7
/proc/31531/stat : 31531 (march_hi) R 31529 31531 31531 0 -1 4202496 3129 0 0 0 32149 3 0 0 25 0 1 0 548678708 13836288 2852 33554432000 4194304 4955518 140737080243728 18446744073709551615 4224768 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31531/statm: 3378 2852 106 186 0 3191 0
[pid=31532] ppid=31505 vsize=416352 CPUtime=0.72 cores=0-7
/proc/31532/stat : 31532 (python) S 31505 31505 30480 0 -1 4202560 872 0 0 0 42 30 0 0 15 0 1 0 548678712 426344448 5892 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 6107919037985947905 0 0 17 4 0 0 0
/proc/31532/statm: 104088 5892 377 364 0 39985 0
[pid=31534] ppid=31532 vsize=13512 CPUtime=321.97 cores=0-7
/proc/31534/stat : 31534 (march_hi) R 31532 31534 31534 0 -1 4202496 3150 0 0 0 32197 0 0 0 25 0 1 0 548678712 13836288 2868 33554432000 4194304 4955518 140736787511600 18446744073709551615 4237804 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31534/statm: 3378 2868 106 186 0 3191 0
[pid=31539] ppid=31505 vsize=426608 CPUtime=0.43 cores=0-7
/proc/31539/stat : 31539 (python) S 31505 31505 30480 0 -1 4202560 840 0 0 0 23 20 0 0 18 0 1 0 548688707 436846592 5909 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 12841715270284181764 0 0 17 4 0 0 0
/proc/31539/statm: 106652 5909 371 364 0 42546 0
[pid=31541] ppid=31539 vsize=181248 CPUtime=222.25 cores=0-7
/proc/31541/stat : 31541 (kcnfs-2006) R 31539 31541 31541 0 -1 4202496 622 0 0 0 22225 0 0 0 25 0 1 0 548688707 185597952 359 33554432000 134512640 135163436 4291129632 18446744073709551615 134564884 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/31541/statm: 45312 359 76 159 0 45152 0
[pid=31543] ppid=31505 vsize=436864 CPUtime=0.49 cores=0-7
/proc/31543/stat : 31543 (python) S 31505 31505 30480 0 -1 4202560 854 0 0 0 26 23 0 0 15 0 1 0 548688715 447348736 5943 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 16060394827405033732 0 0 17 3 0 0 0
/proc/31543/statm: 109216 5943 371 364 0 45107 0
[pid=31545] ppid=31543 vsize=9452 CPUtime=221.46 cores=0-7
/proc/31545/stat : 31545 (march_hi) R 31543 31545 31545 0 -1 4202496 1881 0 0 0 22146 0 0 0 25 0 1 0 548688715 9678848 1600 33554432000 4194304 4955518 140733473893440 18446744073709551615 4237743 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31545/statm: 2363 1600 106 186 0 2176 0
[pid=31505/tid=31512] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31512/stat : 31512 (python) S 31503 31505 30480 0 -1 4202560 29 934 0 0 0 0 0 0 18 0 11 0 548678685 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 4 0 0 0
[pid=31505/tid=31515] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31515/stat : 31515 (python) S 31503 31505 30480 0 -1 4202560 33 934 0 0 0 0 0 0 18 0 11 0 548678689 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 3 0 0 0
[pid=31505/tid=31518] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31518/stat : 31518 (python) S 31503 31505 30480 0 -1 4202560 81 934 0 0 0 0 0 0 19 0 11 0 548678693 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 0 0 0 0
[pid=31505/tid=31521] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31521/stat : 31521 (python) S 31503 31505 30480 0 -1 4202560 32 934 0 0 0 0 0 0 20 0 11 0 548678696 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 0 0 0 0
[pid=31505/tid=31524] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31524/stat : 31524 (python) S 31503 31505 30480 0 -1 4202560 33 934 0 0 0 0 0 0 21 0 11 0 548678700 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 0 0 0 0
[pid=31505/tid=31527] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31527/stat : 31527 (python) S 31503 31505 30480 0 -1 4202560 31 934 0 0 0 0 0 0 23 0 11 0 548678704 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 4 0 0 0
[pid=31505/tid=31530] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31530/stat : 31530 (python) S 31503 31505 30480 0 -1 4202560 33 934 0 0 0 0 0 0 25 0 11 0 548678708 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 4 0 0 0
[pid=31505/tid=31533] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31533/stat : 31533 (python) S 31503 31505 30480 0 -1 4202560 34 934 0 0 0 0 0 0 25 0 11 0 548678712 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 4 0 0 0
[pid=31505/tid=31540] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31540/stat : 31540 (python) S 31503 31505 30480 0 -1 4202560 38 934 0 0 0 0 0 0 20 0 11 0 548688707 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 4 0 0 0
[pid=31505/tid=31544] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31544/stat : 31544 (python) S 31503 31505 30480 0 -1 4202560 34 934 0 0 0 0 0 0 21 0 11 0 548688715 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2574.68
Current children cumulated vsize (KiB) 6540336

[startup+323.502 s]
/proc/loadavg: 8.11 8.18 8.19 9/300 31552
/proc/meminfo: memFree=27976980/32951124 swapFree=67107192/67111528
[pid=31505] ppid=31503 vsize=447108 CPUtime=0.96 cores=0-7
/proc/31505/stat : 31505 (python) S 31503 31505 30480 0 -1 4202496 17748 934 1 0 85 11 0 0 18 0 11 0 548678624 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216474059 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/31505/statm: 111777 7481 1894 364 0 47668 0
[pid=31511] ppid=31505 vsize=288832 CPUtime=0.18 cores=0-7
/proc/31511/stat : 31511 (python) S 31505 31505 30480 0 -1 4202560 1086 0 0 0 10 8 0 0 19 0 2 0 548678685 295763968 5621 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216474059 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/31511/statm: 72208 5621 410 364 0 8126 0
[pid=31513] ppid=31511 vsize=1017800 CPUtime=100.16 cores=0-7
/proc/31513/stat : 31513 (gnovelty+2) T 31511 31513 31513 0 -1 4202496 437 0 0 0 10016 0 0 0 25 0 1 0 548678685 1042227200 172 33554432000 134512640 135663704 4288779728 18446744073709551615 134525132 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/31513/statm: 254450 172 113 282 0 254166 0
[pid=31511/tid=31538] ppid=31505 vsize=288832 CPUtime=0 cores=0-7
/proc/31511/task/31538/stat : 31538 (python) S 31505 31505 30480 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 548688703 295763968 5621 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 514 18446604451036730688 0 0 -1 2 0 0 0
[pid=31514] ppid=31505 vsize=289280 CPUtime=0.72 cores=0-7
/proc/31514/stat : 31514 (python) S 31505 31505 30480 0 -1 4202560 1145 0 0 0 38 34 0 0 15 0 2 0 548678689 296222720 5730 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/31514/statm: 72320 5730 400 364 0 8235 0
[pid=31516] ppid=31514 vsize=13512 CPUtime=321.96 cores=0-7
/proc/31516/stat : 31516 (march_hi) R 31514 31516 31516 0 -1 4202496 3124 0 0 0 32196 0 0 0 25 0 1 0 548678689 13836288 2845 33554432000 4194304 4955518 140736054912704 18446744073709551615 4297490 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31516/statm: 3378 2845 106 186 0 3191 0
[pid=31514/tid=31549] ppid=31505 vsize=289280 CPUtime=0 cores=0-7
/proc/31514/task/31549/stat : 31549 (python) S 31505 31505 30480 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 548698305 296222720 5730 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 514 18446604452936784640 0 0 -1 1 0 0 0
[pid=31517] ppid=31505 vsize=365072 CPUtime=0.18 cores=0-7
/proc/31517/stat : 31517 (python) S 31505 31505 30480 0 -1 4202560 1006 0 0 0 9 9 0 0 18 0 2 0 548678692 373833728 5907 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216474059 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/31517/statm: 91268 5907 400 364 0 27180 0
[pid=31519] ppid=31517 vsize=1017800 CPUtime=100.16 cores=0-7
/proc/31519/stat : 31519 (gnovelty+2) T 31517 31519 31519 0 -1 4202496 423 0 0 0 10016 0 0 0 25 0 1 0 548678693 1042227200 171 33554432000 134512640 135663704 4292807776 18446744073709551615 134524945 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/31519/statm: 254450 171 113 282 0 254166 0
[pid=31517/tid=31542] ppid=31505 vsize=365072 CPUtime=0 cores=0-7
/proc/31517/task/31542/stat : 31542 (python) S 31505 31505 30480 0 -1 4202560 114 0 0 0 0 0 0 0 19 0 2 0 548688710 373833728 5907 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 514 18446604447871087552 0 0 -1 5 0 0 0
[pid=31520] ppid=31505 vsize=375328 CPUtime=0.72 cores=0-7
/proc/31520/stat : 31520 (python) S 31505 31505 30480 0 -1 4202560 818 0 0 0 39 33 0 0 15 0 1 0 548678696 384335872 5853 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 16168331684880548098 0 0 17 6 0 0 0
/proc/31520/statm: 93832 5853 371 364 0 29741 0
[pid=31522] ppid=31520 vsize=13512 CPUtime=322.04 cores=0-7
/proc/31522/stat : 31522 (march_hi) R 31520 31522 31522 0 -1 4202496 3116 0 0 0 32204 0 0 0 25 0 1 0 548678696 13836288 2845 33554432000 4194304 4955518 140736994141952 18446744073709551615 4296823 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31522/statm: 3378 2845 106 186 0 3191 0
[pid=31523] ppid=31505 vsize=385584 CPUtime=0.72 cores=0-7
/proc/31523/stat : 31523 (python) S 31505 31505 30480 0 -1 4202560 817 0 0 0 39 33 0 0 15 0 1 0 548678700 394838016 5874 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 7215461498691223810 0 0 17 7 0 0 0
/proc/31523/statm: 96396 5874 371 364 0 32302 0
[pid=31525] ppid=31523 vsize=13512 CPUtime=317.31 cores=0-7
/proc/31525/stat : 31525 (march_hi) R 31523 31525 31525 0 -1 4202496 3036 0 0 0 31731 0 0 0 25 0 1 0 548678700 13836288 2762 33554432000 4194304 4955518 140735285938032 18446744073709551615 4296850 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31525/statm: 3378 2762 106 186 0 3191 0
[pid=31526] ppid=31505 vsize=395840 CPUtime=0.71 cores=0-7
/proc/31526/stat : 31526 (python) S 31505 31505 30480 0 -1 4202560 833 0 0 0 39 32 0 0 15 0 1 0 548678704 405340160 5845 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 10849356224583926020 0 0 17 1 0 0 0
/proc/31526/statm: 98960 5845 372 364 0 34863 0
[pid=31528] ppid=31526 vsize=13512 CPUtime=320.87 cores=0-7
/proc/31528/stat : 31528 (march_hi) R 31526 31528 31528 0 -1 4202496 3128 0 0 0 32085 2 0 0 25 0 1 0 548678704 13836288 2853 33554432000 4194304 4955518 140734064562608 18446744073709551615 4224406 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31528/statm: 3378 2853 106 186 0 3191 0
[pid=31529] ppid=31505 vsize=406096 CPUtime=0.73 cores=0-7
/proc/31529/stat : 31529 (python) S 31505 31505 30480 0 -1 4202560 830 0 0 0 41 32 0 0 15 0 1 0 548678708 415842304 5968 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 9465660423446429956 0 0 17 0 0 0 0
/proc/31529/statm: 101524 5968 371 364 0 37424 0
[pid=31531] ppid=31529 vsize=13512 CPUtime=321.92 cores=0-7
/proc/31531/stat : 31531 (march_hi) R 31529 31531 31531 0 -1 4202496 3137 0 0 0 32189 3 0 0 25 0 1 0 548678708 13836288 2860 33554432000 4194304 4955518 140737080243728 18446744073709551615 4238087 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31531/statm: 3378 2860 106 186 0 3191 0
[pid=31532] ppid=31505 vsize=416352 CPUtime=0.73 cores=0-7
/proc/31532/stat : 31532 (python) S 31505 31505 30480 0 -1 4202560 872 0 0 0 43 30 0 0 15 0 1 0 548678712 426344448 5892 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 5602231328016662785 0 0 17 4 0 0 0
/proc/31532/statm: 104088 5892 377 364 0 39985 0
[pid=31534] ppid=31532 vsize=13512 CPUtime=322.37 cores=0-7
/proc/31534/stat : 31534 (march_hi) R 31532 31534 31534 0 -1 4202496 3158 0 0 0 32237 0 0 0 25 0 1 0 548678712 13836288 2876 33554432000 4194304 4955518 140736787511600 18446744073709551615 4421280 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31534/statm: 3378 2876 106 186 0 3191 0
[pid=31539] ppid=31505 vsize=426608 CPUtime=0.43 cores=0-7
/proc/31539/stat : 31539 (python) S 31505 31505 30480 0 -1 4202560 840 0 0 0 23 20 0 0 18 0 1 0 548688707 436846592 5909 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 12841715270284181764 0 0 17 4 0 0 0
/proc/31539/statm: 106652 5909 371 364 0 42546 0
[pid=31541] ppid=31539 vsize=181248 CPUtime=222.65 cores=0-7
/proc/31541/stat : 31541 (kcnfs-2006) R 31539 31541 31541 0 -1 4202496 622 0 0 0 22265 0 0 0 25 0 1 0 548688707 185597952 359 33554432000 134512640 135163436 4291129632 18446744073709551615 134602406 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/31541/statm: 45312 359 76 159 0 45152 0
[pid=31543] ppid=31505 vsize=436864 CPUtime=0.49 cores=0-7
/proc/31543/stat : 31543 (python) S 31505 31505 30480 0 -1 4202560 854 0 0 0 26 23 0 0 15 0 1 0 548688715 447348736 5943 33554432000 4194304 5683644 140734210798320 18446744073709551615 210204667462 0 0 16781312 514 16060394827405033732 0 0 17 3 0 0 0
/proc/31543/statm: 109216 5943 371 364 0 45107 0
[pid=31545] ppid=31543 vsize=9452 CPUtime=221.86 cores=0-7
/proc/31545/stat : 31545 (march_hi) R 31543 31545 31545 0 -1 4202496 1884 0 0 0 22186 0 0 0 25 0 1 0 548688715 9678848 1603 33554432000 4194304 4955518 140733473893440 18446744073709551615 4237810 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31545/statm: 2363 1603 106 186 0 2176 0
[pid=31505/tid=31512] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31512/stat : 31512 (python) S 31503 31505 30480 0 -1 4202560 29 934 0 0 0 0 0 0 18 0 11 0 548678685 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 4 0 0 0
[pid=31505/tid=31515] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31515/stat : 31515 (python) S 31503 31505 30480 0 -1 4202560 33 934 0 0 0 0 0 0 18 0 11 0 548678689 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 3 0 0 0
[pid=31505/tid=31518] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31518/stat : 31518 (python) S 31503 31505 30480 0 -1 4202560 81 934 0 0 0 0 0 0 19 0 11 0 548678693 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 0 0 0 0
[pid=31505/tid=31521] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31521/stat : 31521 (python) S 31503 31505 30480 0 -1 4202560 32 934 0 0 0 0 0 0 20 0 11 0 548678696 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 0 0 0 0
[pid=31505/tid=31524] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31524/stat : 31524 (python) S 31503 31505 30480 0 -1 4202560 33 934 0 0 0 0 0 0 21 0 11 0 548678700 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 0 0 0 0
[pid=31505/tid=31527] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31527/stat : 31527 (python) S 31503 31505 30480 0 -1 4202560 31 934 0 0 0 0 0 0 23 0 11 0 548678704 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 4 0 0 0
[pid=31505/tid=31530] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31530/stat : 31530 (python) S 31503 31505 30480 0 -1 4202560 33 934 0 0 0 0 0 0 25 0 11 0 548678708 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 4 0 0 0
[pid=31505/tid=31533] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31533/stat : 31533 (python) S 31503 31505 30480 0 -1 4202560 34 934 0 0 0 0 0 0 25 0 11 0 548678712 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 4 0 0 0
[pid=31505/tid=31540] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31540/stat : 31540 (python) S 31503 31505 30480 0 -1 4202560 38 934 0 0 0 0 0 0 20 0 11 0 548688707 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 4 0 0 0
[pid=31505/tid=31544] ppid=31503 vsize=447108 CPUtime=0 cores=0-7
/proc/31505/task/31544/stat : 31544 (python) S 31503 31505 30480 0 -1 4202560 34 934 0 0 0 0 0 0 21 0 11 0 548688715 457838592 7481 33554432000 4194304 5683644 140734210798320 18446744073709551615 210216470961 0 0 16781312 2 18446604452936782848 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2577.87
Current children cumulated vsize (KiB) 6540336

Child status: 0
Real time (s): 323.599
CPU time (s): 2578.67
CPU user time (s): 2575.85
CPU system time (s): 2.82057
CPU usage (%): 796.871
Max. virtual memory (cumulated for all children) (KiB): 6540336

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2575.85
system time used= 2.82057
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 54633
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 7546
involuntary context switches= 17054

runsolver used 1.50877 second user time and 3.19051 second system time

The end

Launcher Data 

Begin job on node130 at 2011-05-02 23:13:21
IDJOB=3322015
IDBENCH=82793
IDSOLVER=1712
FILE ID=node130/3322015-1304370801
RUNJOBID= node130-1304366466-30494
PBS_JOBID= 13181741
Free space on /tmp= 73460 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/medium/unif-k5-r21.3-v100-c2130-S1760258375-046.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3322015-1304370801/watcher-3322015-1304370801 -o /tmp/evaluation-result-3322015-1304370801/solver-3322015-1304370801 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3322015-1304370801.cnf -seed 1031125028 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= f913d39a64a2142ed34486c38d5636c0
RANDOM SEED=1031125028

node130.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      28107488 kB
Buffers:        273432 kB
Cached:        2518532 kB
SwapCached:       2028 kB
Active:        2358920 kB
Inactive:      2178868 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      28107488 kB
SwapTotal:    67111528 kB
SwapFree:     67107192 kB
Dirty:           32292 kB
Writeback:           0 kB
AnonPages:     1745468 kB
Mapped:          21668 kB
Slab:           222308 kB
PageTables:      18480 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 44262664 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73428 MiB
End job on node130 at 2011-05-02 23:18:45