Trace number 3378310

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? 4793.58 1203.71

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v650-c2769-S1298256564-085.UNKNOWN.cnf
MD5SUMcf48efaad74dd9e258ebbccf8ec047e7
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables650
Number of clauses2769
Sum of the clauses size8307
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 32769
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.38/0.90	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.38/0.90	c INFO: solving HOME/instance-3378310-1305464636.cnf
0.38/0.90	c INFO: computing task features
0.38/0.91	c INFO: computed variable-clause graph statistics
0.38/0.91	c INFO: computed clause balance statistics
0.38/0.96	c INFO: computed variable balance statistics
0.38/0.99	c INFO: computed small-clause counts
0.38/0.99	c INFO: computed Horn-clause counts
0.38/1.00	c INFO: computed variable graph statistics
0.38/1.00	c INFO: collected features for HOME/instance-3378310-1305464636.cnf in 0.10 s
0.38/1.01	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.58)
0.50/1.02	c INFO: running march_hi@0 for 388 with 1199 remaining (b = 0.28)
0.50/1.02	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.07)
0.50/1.04	c INFO: running march_hi@0 for 679 with 1199 remaining (b = 0.36)
399.37/101.24	c INFO: running march_hi@0 for 679 with 1099 remaining (b = 0.33)
399.37/101.27	c INFO: running march_hi@0 for 679 with 1099 remaining (b = 0.29)
1556.70/391.39	c INFO: running kcnfs-2006@0 for 582 with 809 remaining (b = 0.25)
2735.05/686.75	c INFO: running march_hi@0 for 514 with 514 remaining (b = 0.20)
3127.95/785.23	c INFO: running march_hi@0 for 415 with 415 remaining (b = 0.15)
3147.04/790.08	c INFO: running march_hi@0 for 291 with 410 remaining (b = 0.10)
3914.73/982.90	c INFO: running march_hi@389 for 97 with 217 remaining (b = 0.04)
4294.39/1078.35	c INFO: running march_hi@484 for 122 with 122 remaining (b = 0.04)
4318.66/1084.42	c INFO: running march_hi@584 for 116 with 116 remaining (b = 0.04)
4793.02/1203.69	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378310-1305464636/watcher-3378310-1305464636 -o /tmp/evaluation-result-3378310-1305464636/solver-3378310-1305464636 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378310-1305464636.cnf -seed 1111068880 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.29 8.20 7.55 5/359 18748
/proc/meminfo: memFree=18096968/32950928 swapFree=64721848/67111528
[pid=18748] ppid=18746 vsize=15908 CPUtime=0 cores=0,2,4,6
/proc/18748/stat : 18748 (python) R 18746 18748 17961 0 -1 4202496 962 0 0 0 0 0 0 0 25 0 1 0 658063435 16289792 837 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089697936 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/18748/statm: 3977 840 357 364 0 464 0

[startup+0.022689 s]
/proc/loadavg: 8.29 8.20 7.55 5/359 18748
/proc/meminfo: memFree=18096968/32950928 swapFree=64721848/67111528
[pid=18748] ppid=18746 vsize=16396 CPUtime=0.01 cores=0,2,4,6
/proc/18748/stat : 18748 (python) S 18746 18748 17961 0 -1 4202496 1153 0 0 0 1 0 0 0 25 0 1 0 658063435 16789504 1019 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089696256 0 2147483391 16781312 2 18446744071699199166 0 0 17 2 0 0 0
/proc/18748/statm: 4099 1019 413 364 0 586 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16396

[startup+0.100679 s]
/proc/loadavg: 8.29 8.20 7.55 5/359 18748
/proc/meminfo: memFree=18096968/32950928 swapFree=64721848/67111528
[pid=18748] ppid=18746 vsize=104220 CPUtime=0.07 cores=0,2,4,6
/proc/18748/stat : 18748 (python) S 18746 18748 17961 0 -1 4202496 3036 0 0 0 5 2 0 0 25 0 1 0 658063435 106721280 2726 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089696256 0 2147483391 16781312 2 18446744071699199166 0 0 17 2 0 0 0
/proc/18748/statm: 26055 2726 847 364 0 1803 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 104220

[startup+0.300641 s]
/proc/loadavg: 8.29 8.20 7.55 5/359 18748
/proc/meminfo: memFree=18096968/32950928 swapFree=64721848/67111528
[pid=18748] ppid=18746 vsize=124236 CPUtime=0.11 cores=0,2,4,6
/proc/18748/stat : 18748 (python) S 18746 18748 17961 0 -1 4202496 4312 0 1 0 8 3 0 0 25 0 1 0 658063435 127217664 3880 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089698048 0 0 16781312 2 18446744071562362840 0 0 17 2 0 0 0
/proc/18748/statm: 31059 3880 997 364 0 2870 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 124236

[startup+0.700571 s]
/proc/loadavg: 8.29 8.20 7.55 5/359 18748
/proc/meminfo: memFree=18096968/32950928 swapFree=64721848/67111528
[pid=18748] ppid=18746 vsize=135748 CPUtime=0.21 cores=0,2,4,6
/proc/18748/stat : 18748 (python) S 18746 18748 17961 0 -1 4202496 6008 935 1 0 14 7 0 0 22 0 1 0 658063435 139005952 4171 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089696256 0 2147483391 16781312 2 18446744071699199166 0 0 17 2 0 0 0
/proc/18748/statm: 33937 4171 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 135748

[startup+1.50043 s]
/proc/loadavg: 8.29 8.20 7.55 6/360 18754
/proc/meminfo: memFree=18073260/32950928 swapFree=64721848/67111528
[pid=18748] ppid=18746 vsize=320272 CPUtime=0.52 cores=0,2,4,6
/proc/18748/stat : 18748 (python) S 18746 18748 17961 0 -1 4202496 12175 935 1 0 40 12 0 0 25 0 5 0 658063435 327958528 7257 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/18748/statm: 80068 7257 1873 364 0 15977 0
Current children cumulated CPU time (s) 0.52
Current children cumulated vsize (KiB) 320272

[startup+3.10115 s]
/proc/loadavg: 8.29 8.20 7.55 9/372 18766
/proc/meminfo: memFree=18055060/32950928 swapFree=64721848/67111528
[pid=18748] ppid=18746 vsize=320272 CPUtime=0.52 cores=0,2,4,6
/proc/18748/stat : 18748 (python) S 18746 18748 17961 0 -1 4202496 12175 935 1 0 40 12 0 0 25 0 5 0 658063435 327958528 7257 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/18748/statm: 80068 7257 1873 364 0 15977 0
[pid=18755] ppid=18748 vsize=278504 CPUtime=0 cores=0,2,4,6
/proc/18755/stat : 18755 (python) S 18748 18748 17961 0 -1 4202560 732 0 0 0 0 0 0 0 18 0 1 0 658063536 285188096 5561 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718287 0 0 16781312 514 4113371135217729793 0 0 17 2 0 0 0
/proc/18755/statm: 69626 5561 354 364 0 5544 0
[pid=18757] ppid=18755 vsize=1017940 CPUtime=2.08 cores=0,2,4,6
/proc/18757/stat : 18757 (gnovelty+2) R 18755 18757 18757 0 -1 4202496 453 0 0 0 208 0 0 0 25 0 1 0 658063537 1042370560 191 33554432000 134512640 135663704 4289239968 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/18757/statm: 254485 191 113 282 0 254201 0
[pid=18758] ppid=18748 vsize=289380 CPUtime=0 cores=0,2,4,6
/proc/18758/stat : 18758 (python) S 18748 18748 17961 0 -1 4202560 746 0 0 0 0 0 0 0 15 0 1 0 658063537 296325120 5703 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718342 0 0 16781312 514 3884992399646097665 0 0 17 2 0 0 0
/proc/18758/statm: 72345 5703 348 364 0 8260 0
[pid=18760] ppid=18758 vsize=3056 CPUtime=2.07 cores=0,2,4,6
/proc/18760/stat : 18760 (march_hi) R 18758 18760 18760 0 -1 4202496 810 0 0 0 207 0 0 0 25 0 1 0 658063538 3129344 546 33554432000 4194304 4955518 140735588044624 18446744073709551615 4286536 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/18760/statm: 764 546 104 186 0 577 0
[pid=18761] ppid=18748 vsize=299772 CPUtime=0 cores=0,2,4,6
/proc/18761/stat : 18761 (python) S 18748 18748 17961 0 -1 4202560 742 0 0 0 0 0 0 0 18 0 1 0 658063538 306966528 5724 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718342 0 0 16781312 514 3887065253942362370 0 0 17 6 0 0 0
/proc/18761/statm: 74943 5724 348 364 0 10855 0
[pid=18763] ppid=18761 vsize=1017940 CPUtime=2.06 cores=0,2,4,6
/proc/18763/stat : 18763 (gnovelty+2) R 18761 18763 18763 0 -1 4202496 451 0 0 0 206 0 0 0 25 0 1 0 658063538 1042370560 191 33554432000 134512640 135663704 4288959056 18446744073709551615 134524857 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/18763/statm: 254485 191 113 282 0 254201 0
[pid=18764] ppid=18748 vsize=310028 CPUtime=0 cores=0,2,4,6
/proc/18764/stat : 18764 (python) S 18748 18748 17961 0 -1 4202560 758 0 0 0 0 0 0 0 15 0 1 0 658063539 317468672 5728 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718342 0 0 16781312 514 6463483506222661891 0 0 17 6 0 0 0
/proc/18764/statm: 77507 5728 348 364 0 13416 0
[pid=18766] ppid=18764 vsize=3056 CPUtime=2.04 cores=0,2,4,6
/proc/18766/stat : 18766 (march_hi) R 18764 18766 18766 0 -1 4202496 805 0 0 0 204 0 0 0 25 0 1 0 658063539 3129344 544 33554432000 4194304 4955518 140735485646576 18446744073709551615 4227679 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/18766/statm: 764 544 104 186 0 577 0
[pid=18748/tid=18756] ppid=18746 vsize=320272 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18756/stat : 18756 (python) S 18746 18748 17961 0 -1 4202560 25 935 0 0 0 0 0 0 24 0 5 0 658063536 327958528 7257 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 4 0 0 0
[pid=18748/tid=18759] ppid=18746 vsize=320272 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18759/stat : 18759 (python) S 18746 18748 17961 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 5 0 658063537 327958528 7257 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 2 0 0 0
[pid=18748/tid=18762] ppid=18746 vsize=320272 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18762/stat : 18762 (python) S 18746 18748 17961 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 5 0 658063538 327958528 7257 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 0 0 0 0
[pid=18748/tid=18765] ppid=18746 vsize=320272 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18765/stat : 18765 (python) S 18746 18748 17961 0 -1 4202560 30 935 0 0 0 0 0 0 25 0 5 0 658063539 327958528 7257 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 8.77
Current children cumulated vsize (KiB) 3539948

[startup+6.30058 s]
/proc/loadavg: 8.35 8.22 7.56 10/372 18766
/proc/meminfo: memFree=18052964/32950928 swapFree=64721848/67111528
[pid=18748] ppid=18746 vsize=320272 CPUtime=0.52 cores=0,2,4,6
/proc/18748/stat : 18748 (python) S 18746 18748 17961 0 -1 4202496 12175 935 1 0 40 12 0 0 25 0 5 0 658063435 327958528 7257 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/18748/statm: 80068 7257 1873 364 0 15977 0
[pid=18755] ppid=18748 vsize=278504 CPUtime=0 cores=0,2,4,6
/proc/18755/stat : 18755 (python) S 18748 18748 17961 0 -1 4202560 889 0 0 0 0 0 0 0 18 0 1 0 658063536 285188096 5584 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718287 0 0 16781312 514 4113371135217729793 0 0 17 2 0 0 0
/proc/18755/statm: 69626 5584 377 364 0 5544 0
[pid=18757] ppid=18755 vsize=1017940 CPUtime=5.28 cores=0,2,4,6
/proc/18757/stat : 18757 (gnovelty+2) R 18755 18757 18757 0 -1 4202496 453 0 0 0 528 0 0 0 25 0 1 0 658063537 1042370560 191 33554432000 134512640 135663704 4289239968 18446744073709551615 134524988 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/18757/statm: 254485 191 113 282 0 254201 0
[pid=18758] ppid=18748 vsize=289380 CPUtime=0.01 cores=0,2,4,6
/proc/18758/stat : 18758 (python) S 18748 18748 17961 0 -1 4202560 915 0 0 0 0 1 0 0 15 0 1 0 658063537 296325120 5726 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718342 0 0 16781312 514 3884992399646097665 0 0 17 2 0 0 0
/proc/18758/statm: 72345 5726 371 364 0 8260 0
[pid=18760] ppid=18758 vsize=3772 CPUtime=5.24 cores=0,2,4,6
/proc/18760/stat : 18760 (march_hi) R 18758 18760 18760 0 -1 4202496 955 0 0 0 524 0 0 0 25 0 1 0 658063538 3862528 691 33554432000 4194304 4955518 140735588044624 18446744073709551615 4227060 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/18760/statm: 943 691 104 186 0 756 0
[pid=18761] ppid=18748 vsize=299772 CPUtime=0 cores=0,2,4,6
/proc/18761/stat : 18761 (python) S 18748 18748 17961 0 -1 4202560 898 0 0 0 0 0 0 0 18 0 1 0 658063538 306966528 5747 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718342 0 0 16781312 514 3887065253942362370 0 0 17 6 0 0 0
/proc/18761/statm: 74943 5747 371 364 0 10855 0
[pid=18763] ppid=18761 vsize=1017940 CPUtime=5.26 cores=0,2,4,6
/proc/18763/stat : 18763 (gnovelty+2) R 18761 18763 18763 0 -1 4202496 451 0 0 0 526 0 0 0 25 0 1 0 658063538 1042370560 191 33554432000 134512640 135663704 4288959056 18446744073709551615 134544183 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/18763/statm: 254485 191 113 282 0 254201 0
[pid=18764] ppid=18748 vsize=310028 CPUtime=0.01 cores=0,2,4,6
/proc/18764/stat : 18764 (python) S 18748 18748 17961 0 -1 4202560 913 0 0 0 0 1 0 0 15 0 1 0 658063539 317468672 5751 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718342 0 0 16781312 514 6463483506222661891 0 0 17 6 0 0 0
/proc/18764/statm: 77507 5751 371 364 0 13416 0
[pid=18766] ppid=18764 vsize=3772 CPUtime=5.18 cores=0,2,4,6
/proc/18766/stat : 18766 (march_hi) R 18764 18766 18766 0 -1 4202496 947 0 0 0 518 0 0 0 25 0 1 0 658063539 3862528 686 33554432000 4194304 4955518 140735485646576 18446744073709551615 4235311 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/18766/statm: 943 686 104 186 0 756 0
[pid=18748/tid=18756] ppid=18746 vsize=320272 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18756/stat : 18756 (python) S 18746 18748 17961 0 -1 4202560 25 935 0 0 0 0 0 0 24 0 5 0 658063536 327958528 7257 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 4 0 0 0
[pid=18748/tid=18759] ppid=18746 vsize=320272 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18759/stat : 18759 (python) S 18746 18748 17961 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 5 0 658063537 327958528 7257 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 2 0 0 0
[pid=18748/tid=18762] ppid=18746 vsize=320272 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18762/stat : 18762 (python) S 18746 18748 17961 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 5 0 658063538 327958528 7257 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 0 0 0 0
[pid=18748/tid=18765] ppid=18746 vsize=320272 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18765/stat : 18765 (python) S 18746 18748 17961 0 -1 4202560 30 935 0 0 0 0 0 0 25 0 5 0 658063539 327958528 7257 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 21.5
Current children cumulated vsize (KiB) 3541380

[startup+12.7005 s]
/proc/loadavg: 8.32 8.21 7.56 10/373 18768
/proc/meminfo: memFree=18047484/32950928 swapFree=64721848/67111528
[pid=18748] ppid=18746 vsize=320272 CPUtime=0.52 cores=0,2,4,6
/proc/18748/stat : 18748 (python) S 18746 18748 17961 0 -1 4202496 12175 935 1 0 40 12 0 0 25 0 5 0 658063435 327958528 7257 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/18748/statm: 80068 7257 1873 364 0 15977 0
[pid=18755] ppid=18748 vsize=278504 CPUtime=0.02 cores=0,2,4,6
/proc/18755/stat : 18755 (python) S 18748 18748 17961 0 -1 4202560 892 0 0 0 1 1 0 0 18 0 1 0 658063536 285188096 5584 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718287 0 0 16781312 514 11825432274684313857 0 0 17 2 0 0 0
/proc/18755/statm: 69626 5584 377 364 0 5544 0
[pid=18757] ppid=18755 vsize=1017940 CPUtime=11.68 cores=0,2,4,6
/proc/18757/stat : 18757 (gnovelty+2) R 18755 18757 18757 0 -1 4202496 453 0 0 0 1168 0 0 0 25 0 1 0 658063537 1042370560 191 33554432000 134512640 135663704 4289239968 18446744073709551615 134524509 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/18757/statm: 254485 191 113 282 0 254201 0
[pid=18758] ppid=18748 vsize=289380 CPUtime=0.02 cores=0,2,4,6
/proc/18758/stat : 18758 (python) S 18748 18748 17961 0 -1 4202560 923 0 0 0 0 2 0 0 15 0 1 0 658063537 296325120 5726 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718342 0 0 16781312 514 11825432274684313857 0 0 17 2 0 0 0
/proc/18758/statm: 72345 5726 371 364 0 8260 0
[pid=18760] ppid=18758 vsize=5196 CPUtime=11.61 cores=0,2,4,6
/proc/18760/stat : 18760 (march_hi) R 18758 18760 18760 0 -1 4202496 1206 0 0 0 1161 0 0 0 25 0 1 0 658063538 5320704 942 33554432000 4194304 4955518 140735588044624 18446744073709551615 4224658 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/18760/statm: 1299 942 104 186 0 1112 0
[pid=18761] ppid=18748 vsize=299772 CPUtime=0.02 cores=0,2,4,6
/proc/18761/stat : 18761 (python) S 18748 18748 17961 0 -1 4202560 902 0 0 0 2 0 0 0 18 0 1 0 658063538 306966528 5747 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718342 0 0 16781312 514 11825432274684313858 0 0 17 6 0 0 0
/proc/18761/statm: 74943 5747 371 364 0 10855 0
[pid=18763] ppid=18761 vsize=1017940 CPUtime=11.66 cores=0,2,4,6
/proc/18763/stat : 18763 (gnovelty+2) R 18761 18763 18763 0 -1 4202496 451 0 0 0 1166 0 0 0 25 0 1 0 658063538 1042370560 191 33554432000 134512640 135663704 4288959056 18446744073709551615 134545644 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/18763/statm: 254485 191 113 282 0 254201 0
[pid=18764] ppid=18748 vsize=310028 CPUtime=0.02 cores=0,2,4,6
/proc/18764/stat : 18764 (python) S 18748 18748 17961 0 -1 4202560 920 0 0 0 1 1 0 0 15 0 1 0 658063539 317468672 5751 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718342 0 0 16781312 514 11825432274684313859 0 0 17 6 0 0 0
/proc/18764/statm: 77507 5751 371 364 0 13416 0
[pid=18766] ppid=18764 vsize=5196 CPUtime=11.43 cores=0,2,4,6
/proc/18766/stat : 18766 (march_hi) R 18764 18766 18766 0 -1 4202496 1193 0 0 0 1143 0 0 0 25 0 1 0 658063539 5320704 932 33554432000 4194304 4955518 140735485646576 18446744073709551615 4314117 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/18766/statm: 1299 932 104 186 0 1112 0
[pid=18748/tid=18756] ppid=18746 vsize=320272 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18756/stat : 18756 (python) S 18746 18748 17961 0 -1 4202560 25 935 0 0 0 0 0 0 24 0 5 0 658063536 327958528 7257 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 4 0 0 0
[pid=18748/tid=18759] ppid=18746 vsize=320272 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18759/stat : 18759 (python) S 18746 18748 17961 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 5 0 658063537 327958528 7257 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 2 0 0 0
[pid=18748/tid=18762] ppid=18746 vsize=320272 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18762/stat : 18762 (python) S 18746 18748 17961 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 5 0 658063538 327958528 7257 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 0 0 0 0
[pid=18748/tid=18765] ppid=18746 vsize=320272 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18765/stat : 18765 (python) S 18746 18748 17961 0 -1 4202560 30 935 0 0 0 0 0 0 25 0 5 0 658063539 327958528 7257 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 46.98
Current children cumulated vsize (KiB) 3544228

[startup+25.5012 s]
/proc/loadavg: 8.47 8.25 7.58 10/373 18768
/proc/meminfo: memFree=18019696/32950928 swapFree=64721848/67111528
[pid=18748] ppid=18746 vsize=320272 CPUtime=0.52 cores=0,2,4,6

################
# More data... #
################

[pid=18761/tid=18829] ppid=18748 vsize=299772 CPUtime=0 cores=0,2,4,6
/proc/18761/task/18829/stat : 18829 (python) S 18748 18748 17961 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 658073562 306966528 5777 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 514 18446604453817721152 0 0 -1 6 0 0 0
[pid=18764] ppid=18748 vsize=310028 CPUtime=1.74 cores=0,2,4,6
/proc/18764/stat : 18764 (python) S 18748 18748 17961 0 -1 4202560 1150 0 0 0 92 82 0 0 16 0 2 0 658063539 317468672 5781 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/18764/statm: 77507 5781 400 364 0 13416 0
[pid=18766] ppid=18764 vsize=110392 CPUtime=683.34 cores=0,2,4,6
/proc/18766/stat : 18766 (march_hi) T 18764 18766 18766 0 -1 4202496 24671 0 0 0 68329 5 0 0 25 0 1 0 658063539 113041408 24410 33554432000 4194304 4955518 140735485646576 18446744073709551615 4230024 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/18766/statm: 27598 24410 104 186 0 27411 0
[pid=18764/tid=18977] ppid=18748 vsize=310028 CPUtime=0 cores=0,2,4,6
/proc/18764/task/18977/stat : 18977 (python) S 18748 18748 17961 0 -1 4202560 45 0 0 0 0 0 0 0 16 0 2 0 658132109 317468672 5781 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 514 18446604449741711552 0 0 -1 6 0 0 0
[pid=18826] ppid=18748 vsize=320284 CPUtime=1.75 cores=0,2,4,6
/proc/18826/stat : 18826 (python) S 18748 18748 17961 0 -1 4202560 1153 0 0 0 88 87 0 0 16 0 2 0 658073560 327970816 5802 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/18826/statm: 80071 5802 400 364 0 15977 0
[pid=18828] ppid=18826 vsize=110392 CPUtime=683.6 cores=0,2,4,6
/proc/18828/stat : 18828 (march_hi) T 18826 18828 18828 0 -1 4202496 24667 0 0 0 68354 6 0 0 25 0 1 0 658073560 113041408 24406 33554432000 4194304 4955518 140736816743312 18446744073709551615 4227565 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/18828/statm: 27598 24406 104 186 0 27411 0
[pid=18826/tid=18988] ppid=18748 vsize=320284 CPUtime=0 cores=0,2,4,6
/proc/18826/task/18988/stat : 18988 (python) S 18748 18748 17961 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 658141958 327970816 5802 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 514 18446604443751454720 0 0 -1 0 0 0 0
[pid=18830] ppid=18748 vsize=330684 CPUtime=2.32 cores=0,2,4,6
/proc/18830/stat : 18830 (python) S 18748 18748 17961 0 -1 4202560 1151 0 0 0 121 111 0 0 15 0 2 0 658073563 338620416 5825 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718342 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18830/statm: 82671 5825 400 364 0 18574 0
[pid=18832] ppid=18830 vsize=173580 CPUtime=902.68 cores=0,2,4,6
/proc/18832/stat : 18832 (march_hi) R 18830 18832 18832 0 -1 4202496 31851 0 0 0 90261 7 0 0 25 0 1 0 658073563 177745920 31591 33554432000 4194304 4955518 140733948233312 18446744073709551615 4224608 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/18832/statm: 43395 31591 104 186 0 43208 0
[pid=18830/tid=18992] ppid=18748 vsize=330684 CPUtime=0 cores=0,2,4,6
/proc/18830/task/18992/stat : 18992 (python) S 18748 18748 17961 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 658142442 338620416 5825 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 514 18446604443751455616 0 0 -1 4 0 0 0
[pid=18955] ppid=18748 vsize=340940 CPUtime=1.5 cores=0,2,4,6
/proc/18955/stat : 18955 (python) S 18748 18748 17961 0 -1 4202560 1153 0 0 0 79 71 0 0 19 0 2 0 658102574 349122560 5849 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/18955/statm: 85235 5849 400 364 0 21135 0
[pid=18957] ppid=18955 vsize=182580 CPUtime=584.62 cores=0,2,4,6
/proc/18957/stat : 18957 (kcnfs-2006) T 18955 18957 18957 0 -1 4202496 648 0 0 0 58462 0 0 0 25 0 1 0 658102575 186961920 389 33554432000 134512640 135163436 4290493216 18446744073709551615 134559186 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/18957/statm: 45645 389 77 159 0 45485 0
[pid=18955/tid=19004] ppid=18748 vsize=340940 CPUtime=0 cores=0,2,4,6
/proc/18955/task/19004/stat : 19004 (python) S 18748 18748 17961 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 658161725 349122560 5849 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 514 18446604453846581696 0 0 -1 6 0 0 0
[pid=18978] ppid=18748 vsize=351196 CPUtime=1.34 cores=0,2,4,6
/proc/18978/stat : 18978 (python) S 18748 18748 17961 0 -1 4202560 1059 0 0 0 72 62 0 0 15 0 1 0 658132110 359624704 5829 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718342 0 0 16781312 514 11825432274684313857 0 0 17 6 0 0 0
/proc/18978/statm: 87799 5829 377 364 0 23696 0
[pid=18980] ppid=18978 vsize=80924 CPUtime=514.75 cores=0,2,4,6
/proc/18980/stat : 18980 (march_hi) R 18978 18980 18980 0 -1 4202496 18809 0 0 0 51471 4 0 0 25 0 1 0 658132111 82866176 18541 33554432000 4194304 4955518 140734614308368 18446744073709551615 4209635 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/18980/statm: 20231 18541 104 186 0 20044 0
[pid=18989] ppid=18748 vsize=361056 CPUtime=1.1 cores=0,2,4,6
/proc/18989/stat : 18989 (python) S 18748 18748 17961 0 -1 4202560 1040 0 0 0 56 54 0 0 15 0 1 0 658141959 369721344 5746 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718342 0 0 16781312 514 11825432274684313860 0 0 17 2 0 0 0
/proc/18989/statm: 90264 5746 377 364 0 26158 0
[pid=18991] ppid=18989 vsize=80924 CPUtime=411.41 cores=0,2,4,6
/proc/18991/stat : 18991 (march_hi) R 18989 18991 18991 0 -1 4202496 15462 0 0 0 41139 2 0 0 25 0 1 0 658141959 82866176 15194 33554432000 4194304 4955518 140737289592160 18446744073709551615 4227502 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/18991/statm: 20231 15194 104 186 0 20044 0
[pid=18748/tid=18756] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18756/stat : 18756 (python) S 18746 18748 17961 0 -1 4202560 25 935 0 0 0 0 0 0 24 0 10 0 658063536 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 4 0 0 0
[pid=18748/tid=18759] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18759/stat : 18759 (python) S 18746 18748 17961 0 -1 4202560 30 935 0 0 0 0 0 0 18 0 10 0 658063537 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 2 0 0 0
[pid=18748/tid=18762] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18762/stat : 18762 (python) S 18746 18748 17961 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 10 0 658063538 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 0 0 0 0
[pid=18748/tid=18765] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18765/stat : 18765 (python) S 18746 18748 17961 0 -1 4202560 30 935 0 0 0 0 0 0 25 0 10 0 658063539 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 6 0 0 0
[pid=18748/tid=18827] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18827/stat : 18827 (python) S 18746 18748 17961 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 10 0 658073560 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 6 0 0 0
[pid=18748/tid=18831] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18831/stat : 18831 (python) S 18746 18748 17961 0 -1 4202560 27 935 0 0 0 0 0 0 18 0 10 0 658073563 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 6 0 0 0
[pid=18748/tid=18956] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18956/stat : 18956 (python) S 18746 18748 17961 0 -1 4202560 31 935 0 0 0 0 0 0 20 0 10 0 658102575 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 6 0 0 0
[pid=18748/tid=18979] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18979/stat : 18979 (python) S 18746 18748 17961 0 -1 4202560 28 935 0 0 0 0 0 0 20 0 10 0 658132110 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 6 0 0 0
[pid=18748/tid=18990] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18990/stat : 18990 (python) S 18746 18748 17961 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 10 0 658141959 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4791.03
Current children cumulated vsize (KiB) 6212284

[startup+1203.5 s]
/proc/loadavg: 8.28 8.23 8.00 9/406 19010
/proc/meminfo: memFree=17448144/32950928 swapFree=64721848/67111528
[pid=18748] ppid=18746 vsize=371944 CPUtime=0.59 cores=0,2,4,6
/proc/18748/stat : 18748 (python) S 18746 18748 17961 0 -1 4202496 17015 935 1 0 46 13 0 0 18 0 10 0 658063435 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/18748/statm: 92986 7394 1892 364 0 28880 0
[pid=18755] ppid=18748 vsize=288748 CPUtime=0.23 cores=0,2,4,6
/proc/18755/stat : 18755 (python) S 18748 18748 17961 0 -1 4202560 1157 0 0 0 12 11 0 0 19 0 2 0 658063536 295677952 5628 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/18755/statm: 72187 5628 416 364 0 8105 0
[pid=18757] ppid=18755 vsize=1017940 CPUtime=100.2 cores=0,2,4,6
/proc/18757/stat : 18757 (gnovelty+2) T 18755 18757 18757 0 -1 4202496 453 0 0 0 10020 0 0 0 25 0 1 0 658063537 1042370560 191 33554432000 134512640 135663704 4289239968 18446744073709551615 134525286 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/18757/statm: 254485 191 113 282 0 254201 0
[pid=18755/tid=18825] ppid=18748 vsize=288748 CPUtime=0 cores=0,2,4,6
/proc/18755/task/18825/stat : 18825 (python) S 18748 18748 17961 0 -1 4202560 42 0 0 0 0 0 0 0 19 0 2 0 658073559 295677952 5628 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 514 18446604453870412096 0 0 -1 2 0 0 0
[pid=18758] ppid=18748 vsize=289380 CPUtime=2.03 cores=0,2,4,6
/proc/18758/stat : 18758 (python) S 18748 18748 17961 0 -1 4202560 1149 0 0 0 101 102 0 0 15 0 2 0 658063537 296325120 5756 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718342 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/18758/statm: 72345 5756 400 364 0 8260 0
[pid=18760] ppid=18758 vsize=173580 CPUtime=797.76 cores=0,2,4,6
/proc/18760/stat : 18760 (march_hi) R 18758 18760 18760 0 -1 4202496 28507 0 0 0 79768 8 0 0 25 0 1 0 658063538 177745920 28243 33554432000 4194304 4955518 140735588044624 18446744073709551615 4298048 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/18760/statm: 43395 28243 104 186 0 43208 0
[pid=18758/tid=18954] ppid=18748 vsize=289380 CPUtime=0 cores=0,2,4,6
/proc/18758/task/18954/stat : 18954 (python) S 18748 18748 17961 0 -1 4202560 45 0 0 0 0 0 0 0 15 0 2 0 658102574 296325120 5756 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 514 18446604448023616320 0 0 -1 2 0 0 0
[pid=18761] ppid=18748 vsize=299772 CPUtime=0.23 cores=0,2,4,6
/proc/18761/stat : 18761 (python) S 18748 18748 17961 0 -1 4202560 1088 0 0 0 14 9 0 0 19 0 2 0 658063538 306966528 5777 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/18761/statm: 74943 5777 400 364 0 10855 0
[pid=18763] ppid=18761 vsize=1017940 CPUtime=100.23 cores=0,2,4,6
/proc/18763/stat : 18763 (gnovelty+2) T 18761 18763 18763 0 -1 4202496 451 0 0 0 10023 0 0 0 25 0 1 0 658063538 1042370560 191 33554432000 134512640 135663704 4288959056 18446744073709551615 134544117 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/18763/statm: 254485 191 113 282 0 254201 0
[pid=18761/tid=18829] ppid=18748 vsize=299772 CPUtime=0 cores=0,2,4,6
/proc/18761/task/18829/stat : 18829 (python) S 18748 18748 17961 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 658073562 306966528 5777 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 514 18446604453817721152 0 0 -1 6 0 0 0
[pid=18764] ppid=18748 vsize=310028 CPUtime=1.74 cores=0,2,4,6
/proc/18764/stat : 18764 (python) S 18748 18748 17961 0 -1 4202560 1150 0 0 0 92 82 0 0 16 0 2 0 658063539 317468672 5781 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/18764/statm: 77507 5781 400 364 0 13416 0
[pid=18766] ppid=18764 vsize=110392 CPUtime=683.34 cores=0,2,4,6
/proc/18766/stat : 18766 (march_hi) T 18764 18766 18766 0 -1 4202496 24671 0 0 0 68329 5 0 0 25 0 1 0 658063539 113041408 24410 33554432000 4194304 4955518 140735485646576 18446744073709551615 4230024 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/18766/statm: 27598 24410 104 186 0 27411 0
[pid=18764/tid=18977] ppid=18748 vsize=310028 CPUtime=0 cores=0,2,4,6
/proc/18764/task/18977/stat : 18977 (python) S 18748 18748 17961 0 -1 4202560 45 0 0 0 0 0 0 0 16 0 2 0 658132109 317468672 5781 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 514 18446604449741711552 0 0 -1 6 0 0 0
[pid=18826] ppid=18748 vsize=320284 CPUtime=1.75 cores=0,2,4,6
/proc/18826/stat : 18826 (python) S 18748 18748 17961 0 -1 4202560 1153 0 0 0 88 87 0 0 16 0 2 0 658073560 327970816 5802 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/18826/statm: 80071 5802 400 364 0 15977 0
[pid=18828] ppid=18826 vsize=110392 CPUtime=683.6 cores=0,2,4,6
/proc/18828/stat : 18828 (march_hi) T 18826 18828 18828 0 -1 4202496 24667 0 0 0 68354 6 0 0 25 0 1 0 658073560 113041408 24406 33554432000 4194304 4955518 140736816743312 18446744073709551615 4227565 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/18828/statm: 27598 24406 104 186 0 27411 0
[pid=18826/tid=18988] ppid=18748 vsize=320284 CPUtime=0 cores=0,2,4,6
/proc/18826/task/18988/stat : 18988 (python) S 18748 18748 17961 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 658141958 327970816 5802 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 514 18446604443751454720 0 0 -1 0 0 0 0
[pid=18830] ppid=18748 vsize=330684 CPUtime=2.32 cores=0,2,4,6
/proc/18830/stat : 18830 (python) S 18748 18748 17961 0 -1 4202560 1151 0 0 0 121 111 0 0 15 0 2 0 658073563 338620416 5825 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718342 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18830/statm: 82671 5825 400 364 0 18574 0
[pid=18832] ppid=18830 vsize=173580 CPUtime=903.08 cores=0,2,4,6
/proc/18832/stat : 18832 (march_hi) R 18830 18832 18832 0 -1 4202496 31862 0 0 0 90301 7 0 0 25 0 1 0 658073563 177745920 31602 33554432000 4194304 4955518 140733948233312 18446744073709551615 4209688 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/18832/statm: 43395 31602 104 186 0 43208 0
[pid=18830/tid=18992] ppid=18748 vsize=330684 CPUtime=0 cores=0,2,4,6
/proc/18830/task/18992/stat : 18992 (python) S 18748 18748 17961 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 658142442 338620416 5825 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 514 18446604443751455616 0 0 -1 4 0 0 0
[pid=18955] ppid=18748 vsize=340940 CPUtime=1.5 cores=0,2,4,6
/proc/18955/stat : 18955 (python) S 18748 18748 17961 0 -1 4202560 1153 0 0 0 79 71 0 0 19 0 2 0 658102574 349122560 5849 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/18955/statm: 85235 5849 400 364 0 21135 0
[pid=18957] ppid=18955 vsize=182580 CPUtime=584.62 cores=0,2,4,6
/proc/18957/stat : 18957 (kcnfs-2006) T 18955 18957 18957 0 -1 4202496 648 0 0 0 58462 0 0 0 25 0 1 0 658102575 186961920 389 33554432000 134512640 135163436 4290493216 18446744073709551615 134559186 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/18957/statm: 45645 389 77 159 0 45485 0
[pid=18955/tid=19004] ppid=18748 vsize=340940 CPUtime=0 cores=0,2,4,6
/proc/18955/task/19004/stat : 19004 (python) S 18748 18748 17961 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 658161725 349122560 5849 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 514 18446604453846581696 0 0 -1 6 0 0 0
[pid=18978] ppid=18748 vsize=351196 CPUtime=1.34 cores=0,2,4,6
/proc/18978/stat : 18978 (python) S 18748 18748 17961 0 -1 4202560 1059 0 0 0 72 62 0 0 15 0 1 0 658132110 359624704 5829 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718342 0 0 16781312 514 11825432274684313857 0 0 17 6 0 0 0
/proc/18978/statm: 87799 5829 377 364 0 23696 0
[pid=18980] ppid=18978 vsize=80924 CPUtime=515.15 cores=0,2,4,6
/proc/18980/stat : 18980 (march_hi) R 18978 18980 18980 0 -1 4202496 18822 0 0 0 51511 4 0 0 25 0 1 0 658132111 82866176 18554 33554432000 4194304 4955518 140734614308368 18446744073709551615 4224687 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/18980/statm: 20231 18554 104 186 0 20044 0
[pid=18989] ppid=18748 vsize=361056 CPUtime=1.1 cores=0,2,4,6
/proc/18989/stat : 18989 (python) S 18748 18748 17961 0 -1 4202560 1040 0 0 0 56 54 0 0 15 0 1 0 658141959 369721344 5746 33554432000 4194304 5683644 140734219706976 18446744073709551615 248089718342 0 0 16781312 514 11825432274684313860 0 0 17 2 0 0 0
/proc/18989/statm: 90264 5746 377 364 0 26158 0
[pid=18991] ppid=18989 vsize=80924 CPUtime=411.81 cores=0,2,4,6
/proc/18991/stat : 18991 (march_hi) R 18989 18991 18991 0 -1 4202496 15475 0 0 0 41179 2 0 0 25 0 1 0 658141959 82866176 15207 33554432000 4194304 4955518 140737289592160 18446744073709551615 4227502 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/18991/statm: 20231 15207 104 186 0 20044 0
[pid=18748/tid=18756] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18756/stat : 18756 (python) S 18746 18748 17961 0 -1 4202560 25 935 0 0 0 0 0 0 24 0 10 0 658063536 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 4 0 0 0
[pid=18748/tid=18759] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18759/stat : 18759 (python) S 18746 18748 17961 0 -1 4202560 30 935 0 0 0 0 0 0 18 0 10 0 658063537 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 2 0 0 0
[pid=18748/tid=18762] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18762/stat : 18762 (python) S 18746 18748 17961 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 10 0 658063538 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 0 0 0 0
[pid=18748/tid=18765] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18765/stat : 18765 (python) S 18746 18748 17961 0 -1 4202560 30 935 0 0 0 0 0 0 25 0 10 0 658063539 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 6 0 0 0
[pid=18748/tid=18827] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18827/stat : 18827 (python) S 18746 18748 17961 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 10 0 658073560 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 6 0 0 0
[pid=18748/tid=18831] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18831/stat : 18831 (python) S 18746 18748 17961 0 -1 4202560 27 935 0 0 0 0 0 0 18 0 10 0 658073563 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 6 0 0 0
[pid=18748/tid=18956] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18956/stat : 18956 (python) S 18746 18748 17961 0 -1 4202560 31 935 0 0 0 0 0 0 20 0 10 0 658102575 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 6 0 0 0
[pid=18748/tid=18979] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18979/stat : 18979 (python) S 18746 18748 17961 0 -1 4202560 28 935 0 0 0 0 0 0 20 0 10 0 658132110 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 6 0 0 0
[pid=18748/tid=18990] ppid=18746 vsize=371944 CPUtime=0 cores=0,2,4,6
/proc/18748/task/18990/stat : 18990 (python) S 18746 18748 17961 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 10 0 658141959 380870656 7394 33554432000 4194304 5683644 140734219706976 18446744073709551615 248101521841 0 0 16781312 2 18446604443643451328 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4792.62
Current children cumulated vsize (KiB) 6212284

[startup+1203.7 s]
/proc/loadavg: 8.28 8.23 8.00 9/406 19010
/proc/meminfo: memFree=17448144/32950928 swapFree=64721848/67111528
[pid=18748] ppid=18746 vsize=310360 CPUtime=4793.55 cores=0,2,4,6
/proc/18748/stat : 18748 (python) R 18746 18748 17961 0 -1 4202496 19581 156894 1 0 48 13 478659 635 17 0 1 0 658063435 317808640 7352 33554432000 4194304 5683644 140734219706976 18446744073709551615 5258739 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/18748/statm: 77590 7352 1874 364 0 13514 0
Current children cumulated CPU time (s) 4793.55
Current children cumulated vsize (KiB) 310360

Child status: 0
Real time (s): 1203.71
CPU time (s): 4793.58
CPU user time (s): 4787.09
CPU system time (s): 6.49801
CPU usage (%): 398.233
Max. virtual memory (cumulated for all children) (KiB): 6212284

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4787.09
system time used= 6.49801
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 177970
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 7817
involuntary context switches= 41262

runsolver used 5.04223 second user time and 11.7842 second system time

The end

Launcher Data 

Begin job on node123 at 2011-05-15 15:03:56
IDJOB=3378310
IDBENCH=82719
IDSOLVER=1859
FILE ID=node123/3378310-1305464636
RUNJOBID= node123-1305462800-17979
PBS_JOBID= 13324975
Free space on /tmp= 73380 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v650-c2769-S1298256564-085.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378310-1305464636/watcher-3378310-1305464636 -o /tmp/evaluation-result-3378310-1305464636/solver-3378310-1305464636 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378310-1305464636.cnf -seed 1111068880 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= cf48efaad74dd9e258ebbccf8ec047e7
RANDOM SEED=1111068880

node123.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      18097000 kB
Buffers:        325508 kB
Cached:        4418652 kB
SwapCached:      10528 kB
Active:       11542396 kB
Inactive:      3013156 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      18097000 kB
SwapTotal:    67111528 kB
SwapFree:     64721848 kB
Dirty:           31908 kB
Writeback:           0 kB
AnonPages:     9808900 kB
Mapped:          30576 kB
Slab:           182024 kB
PageTables:      50832 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 62814124 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73340 MiB
End job on node123 at 2011-05-15 15:24:00