Trace number 3378630

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? 4733.01 1198.21

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v700-c2982-S1124205008-074.UNKNOWN.cnf
MD5SUM7363c9bb5019132082aeb3e4836ce4d1
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables700
Number of clauses2982
Sum of the clauses size8946
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 32982
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.28/0.47	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.28/0.47	c INFO: solving HOME/instance-3378630-1305466526.cnf
0.28/0.47	c INFO: computing task features
0.28/0.48	c INFO: computed variable-clause graph statistics
0.28/0.48	c INFO: computed clause balance statistics
0.36/0.53	c INFO: computed variable balance statistics
0.36/0.57	c INFO: computed small-clause counts
0.36/0.57	c INFO: computed Horn-clause counts
0.36/0.58	c INFO: computed variable graph statistics
0.36/0.58	c INFO: collected features for HOME/instance-3378630-1305466526.cnf in 0.11 s
0.36/0.59	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.58)
0.36/0.60	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.07)
0.47/0.61	c INFO: running march_hi@0 for 388 with 1199 remaining (b = 0.29)
0.47/0.62	c INFO: running march_hi@0 for 679 with 1199 remaining (b = 0.35)
381.84/106.87	c INFO: running march_hi@0 for 679 with 1093 remaining (b = 0.31)
406.14/112.90	c INFO: running march_hi@0 for 679 with 1087 remaining (b = 0.28)
1615.91/416.31	c INFO: running kcnfs-2006@0 for 582 with 784 remaining (b = 0.23)
2729.55/695.64	c INFO: running march_hi@0 for 388 with 504 remaining (b = 0.17)
3104.34/789.69	c INFO: running march_hi@0 for 291 with 410 remaining (b = 0.11)
3124.68/794.71	c INFO: running march_hi@101 for 291 with 405 remaining (b = 0.12)
3978.56/1008.91	c INFO: running march_hi@391 for 191 with 191 remaining (b = 0.05)
4267.96/1081.54	c INFO: running march_hi@99 for 118 with 118 remaining (b = 0.04)
4294.63/1088.23	c INFO: running march_hi@679 for 112 with 112 remaining (b = 0.04)
4297.05/1088.85	c INFO: running kcnfs-2006@682 for 111 with 111 remaining (b = 0.04)
4732.46/1198.19	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378630-1305466526/watcher-3378630-1305466526 -o /tmp/evaluation-result-3378630-1305466526/solver-3378630-1305466526 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378630-1305466526.cnf -seed 108187587 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.11 8.01 6.11 3/368 6136
/proc/meminfo: memFree=23707180/32951124 swapFree=44485764/67111528
[pid=6136] ppid=6134 vsize=15908 CPUtime=0 cores=1,3,5,7
/proc/6136/stat : 6136 (python) S 6134 6136 5751 0 -1 4202496 957 0 0 0 0 0 0 0 25 0 1 0 658253070 16289792 834 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446943232 0 2147483391 16781312 2 18446744071699199166 0 0 17 3 0 0 0
/proc/6136/statm: 3977 834 357 364 0 464 0

[startup+0.032229 s]
/proc/loadavg: 8.11 8.01 6.11 3/368 6136
/proc/meminfo: memFree=23707180/32951124 swapFree=44485764/67111528
[pid=6136] ppid=6134 vsize=18884 CPUtime=0.01 cores=1,3,5,7
/proc/6136/stat : 6136 (python) R 6134 6136 5751 0 -1 4202496 1278 0 0 0 1 0 0 0 17 0 1 0 658253070 19337216 1132 33554432000 4194304 5683644 140736125851440 18446744073709551615 4618608 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/6136/statm: 4721 1132 421 364 0 690 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18884

[startup+0.100189 s]
/proc/loadavg: 8.11 8.01 6.11 3/368 6136
/proc/meminfo: memFree=23707180/32951124 swapFree=44485764/67111528
[pid=6136] ppid=6134 vsize=104216 CPUtime=0.06 cores=1,3,5,7
/proc/6136/stat : 6136 (python) S 6134 6136 5751 0 -1 4202496 3034 0 0 0 4 2 0 0 17 0 1 0 658253070 106717184 2725 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446943232 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/6136/statm: 26054 2725 847 364 0 1802 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 104216

[startup+0.300177 s]
/proc/loadavg: 8.11 8.01 6.11 3/368 6136
/proc/meminfo: memFree=23707180/32951124 swapFree=44485764/67111528
[pid=6136] ppid=6134 vsize=168672 CPUtime=0.21 cores=1,3,5,7
/proc/6136/stat : 6136 (python) R 6134 6136 5751 0 -1 4202496 6712 934 1 0 15 6 0 0 17 0 1 0 658253070 172720128 4795 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446600196 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/6136/statm: 42168 4795 1331 364 0 3453 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 168672

[startup+0.701069 s]
/proc/loadavg: 8.11 8.01 6.11 3/368 6136
/proc/meminfo: memFree=23707180/32951124 swapFree=44485764/67111528
[pid=6136] ppid=6134 vsize=320256 CPUtime=0.48 cores=1,3,5,7
/proc/6136/stat : 6136 (python) S 6134 6136 5751 0 -1 4202496 12207 934 1 0 39 9 0 0 20 0 5 0 658253070 327942144 7253 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/6136/statm: 80064 7253 1873 364 0 15973 0
Current children cumulated CPU time (s) 0.48
Current children cumulated vsize (KiB) 320256

[startup+1.50092 s]
/proc/loadavg: 8.11 8.01 6.11 10/398 6181
/proc/meminfo: memFree=23630248/32951124 swapFree=44485764/67111528
[pid=6136] ppid=6134 vsize=320256 CPUtime=0.48 cores=1,3,5,7
/proc/6136/stat : 6136 (python) S 6134 6136 5751 0 -1 4202496 12207 934 1 0 39 9 0 0 20 0 5 0 658253070 327942144 7253 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/6136/statm: 80064 7253 1873 364 0 15973 0
[pid=6158] ppid=6136 vsize=278624 CPUtime=0 cores=1,3,5,7
/proc/6158/stat : 6158 (python) S 6136 6136 5751 0 -1 4202560 731 0 0 0 0 0 0 0 18 0 1 0 658253129 285310976 5569 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965263 0 0 16781312 514 14825964326807928831 0 0 17 3 0 0 0
/proc/6158/statm: 69656 5569 354 364 0 5574 0
[pid=6160] ppid=6158 vsize=1017936 CPUtime=0.51 cores=1,3,5,7
/proc/6160/stat : 6160 (gnovelty+2) R 6158 6160 6160 0 -1 4202496 457 0 0 0 51 0 0 0 20 0 1 0 658253129 1042366464 196 33554432000 134512640 135663704 4290297008 18446744073709551615 134525544 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/6160/statm: 254484 196 113 282 0 254200 0
[pid=6161] ppid=6136 vsize=289396 CPUtime=0 cores=1,3,5,7
/proc/6161/stat : 6161 (python) S 6136 6136 5751 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 658253130 296341504 5707 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965318 0 0 16781312 514 562079694262894591 0 0 17 1 0 0 0
/proc/6161/statm: 72349 5707 348 364 0 8264 0
[pid=6163] ppid=6161 vsize=1017936 CPUtime=0.89 cores=1,3,5,7
/proc/6163/stat : 6163 (gnovelty+2) R 6161 6163 6163 0 -1 4202496 462 0 0 0 89 0 0 0 21 0 1 0 658253130 1042366464 197 33554432000 134512640 135663704 4294649264 18446744073709551615 134524534 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/6163/statm: 254484 197 113 282 0 254200 0
[pid=6164] ppid=6136 vsize=299148 CPUtime=0 cores=1,3,5,7
/proc/6164/stat : 6164 (python) S 6136 6136 5751 0 -1 4202560 748 0 0 0 0 0 0 0 15 0 1 0 658253131 306327552 5600 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965318 0 0 16781312 514 10883010076486893831 0 0 17 3 0 0 0
/proc/6164/statm: 74787 5600 348 364 0 10699 0
[pid=6168] ppid=6164 vsize=2928 CPUtime=0.87 cores=1,3,5,7
/proc/6168/stat : 6168 (march_hi) R 6164 6168 6168 0 -1 4202496 768 0 0 0 87 0 0 0 25 0 1 0 658253132 2998272 508 33554432000 4194304 4955518 140734182633792 18446744073709551615 4285046 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/6168/statm: 732 508 104 186 0 545 0
[pid=6166] ppid=6136 vsize=310012 CPUtime=0 cores=1,3,5,7
/proc/6166/stat : 6166 (python) S 6136 6136 5751 0 -1 4202560 761 0 0 0 0 0 0 0 15 0 1 0 658253132 317452288 5722 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965318 0 0 16781312 514 3605641679400304901 0 0 17 3 0 0 0
/proc/6166/statm: 77503 5722 348 364 0 13412 0
[pid=6169] ppid=6166 vsize=2724 CPUtime=0.37 cores=1,3,5,7
/proc/6169/stat : 6169 (march_hi) R 6166 6169 6169 0 -1 4202496 728 0 0 0 37 0 0 0 24 0 1 0 658253132 2789376 465 33554432000 4194304 4955518 140734434312144 18446744073709551615 4227491 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/6169/statm: 681 465 104 186 0 494 0
[pid=6136/tid=6159] ppid=6134 vsize=320256 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6159/stat : 6159 (python) S 6134 6136 5751 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 5 0 658253129 327942144 7253 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 7 0 0 0
[pid=6136/tid=6162] ppid=6134 vsize=320256 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6162/stat : 6162 (python) S 6134 6136 5751 0 -1 4202560 23 934 0 0 0 0 0 0 21 0 5 0 658253130 327942144 7253 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 1 0 0 0
[pid=6136/tid=6165] ppid=6134 vsize=320256 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6165/stat : 6165 (python) S 6134 6136 5751 0 -1 4202560 35 934 0 0 0 0 0 0 20 0 5 0 658253131 327942144 7253 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
[pid=6136/tid=6167] ppid=6134 vsize=320256 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6167/stat : 6167 (python) S 6134 6136 5751 0 -1 4202560 18 934 0 0 0 0 0 0 21 0 5 0 658253132 327942144 7253 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3.12
Current children cumulated vsize (KiB) 3538960

[startup+3.10062 s]
/proc/loadavg: 8.11 8.01 6.11 10/398 6181
/proc/meminfo: memFree=23626652/32951124 swapFree=44485764/67111528
[pid=6136] ppid=6134 vsize=320256 CPUtime=0.48 cores=1,3,5,7
/proc/6136/stat : 6136 (python) S 6134 6136 5751 0 -1 4202496 12207 934 1 0 39 9 0 0 20 0 5 0 658253070 327942144 7253 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/6136/statm: 80064 7253 1873 364 0 15973 0
[pid=6158] ppid=6136 vsize=278624 CPUtime=0 cores=1,3,5,7
/proc/6158/stat : 6158 (python) S 6136 6136 5751 0 -1 4202560 731 0 0 0 0 0 0 0 18 0 1 0 658253129 285310976 5569 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965263 0 0 16781312 514 14825964326807896326 0 0 17 3 0 0 0
/proc/6158/statm: 69656 5569 354 364 0 5574 0
[pid=6160] ppid=6158 vsize=1017936 CPUtime=1.31 cores=1,3,5,7
/proc/6160/stat : 6160 (gnovelty+2) R 6158 6160 6160 0 -1 4202496 457 0 0 0 131 0 0 0 23 0 1 0 658253129 1042366464 196 33554432000 134512640 135663704 4290297008 18446744073709551615 134525361 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/6160/statm: 254484 196 113 282 0 254200 0
[pid=6161] ppid=6136 vsize=289396 CPUtime=0 cores=1,3,5,7
/proc/6161/stat : 6161 (python) S 6136 6136 5751 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 658253130 296341504 5707 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965318 0 0 16781312 514 562079694262862086 0 0 17 1 0 0 0
/proc/6161/statm: 72349 5707 348 364 0 8264 0
[pid=6163] ppid=6161 vsize=1017936 CPUtime=2.49 cores=1,3,5,7
/proc/6163/stat : 6163 (gnovelty+2) R 6161 6163 6163 0 -1 4202496 462 0 0 0 249 0 0 0 25 0 1 0 658253130 1042366464 197 33554432000 134512640 135663704 4294649264 18446744073709551615 134524897 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/6163/statm: 254484 197 113 282 0 254200 0
[pid=6164] ppid=6136 vsize=299148 CPUtime=0 cores=1,3,5,7
/proc/6164/stat : 6164 (python) S 6136 6136 5751 0 -1 4202560 749 0 0 0 0 0 0 0 15 0 1 0 658253131 306327552 5600 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965318 0 0 16781312 514 9656781382877183999 0 0 17 3 0 0 0
/proc/6164/statm: 74787 5600 348 364 0 10699 0
[pid=6168] ppid=6164 vsize=3180 CPUtime=2.47 cores=1,3,5,7
/proc/6168/stat : 6168 (march_hi) R 6164 6168 6168 0 -1 4202496 842 0 0 0 247 0 0 0 25 0 1 0 658253132 3256320 582 33554432000 4194304 4955518 140734182633792 18446744073709551615 4227667 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/6168/statm: 795 582 104 186 0 608 0
[pid=6166] ppid=6136 vsize=310012 CPUtime=0 cores=1,3,5,7
/proc/6166/stat : 6166 (python) S 6136 6136 5751 0 -1 4202560 761 0 0 0 0 0 0 0 15 0 1 0 658253132 317452288 5722 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965318 0 0 16781312 514 3605641679400304901 0 0 17 3 0 0 0
/proc/6166/statm: 77503 5722 348 364 0 13412 0
[pid=6169] ppid=6166 vsize=2928 CPUtime=1.16 cores=1,3,5,7
/proc/6169/stat : 6169 (march_hi) R 6166 6169 6169 0 -1 4202496 785 0 0 0 116 0 0 0 25 0 1 0 658253132 2998272 522 33554432000 4194304 4955518 140734434312144 18446744073709551615 4227741 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/6169/statm: 732 522 104 186 0 545 0
[pid=6136/tid=6159] ppid=6134 vsize=320256 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6159/stat : 6159 (python) S 6134 6136 5751 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 5 0 658253129 327942144 7253 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 7 0 0 0
[pid=6136/tid=6162] ppid=6134 vsize=320256 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6162/stat : 6162 (python) S 6134 6136 5751 0 -1 4202560 23 934 0 0 0 0 0 0 21 0 5 0 658253130 327942144 7253 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 1 0 0 0
[pid=6136/tid=6165] ppid=6134 vsize=320256 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6165/stat : 6165 (python) S 6134 6136 5751 0 -1 4202560 35 934 0 0 0 0 0 0 20 0 5 0 658253131 327942144 7253 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
[pid=6136/tid=6167] ppid=6134 vsize=320256 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6167/stat : 6167 (python) S 6134 6136 5751 0 -1 4202560 18 934 0 0 0 0 0 0 21 0 5 0 658253132 327942144 7253 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 7.91
Current children cumulated vsize (KiB) 3539416

[startup+6.30103 s]
/proc/loadavg: 8.18 8.02 6.12 7/386 6213
/proc/meminfo: memFree=23642436/32951124 swapFree=44485764/67111528
[pid=6136] ppid=6134 vsize=320256 CPUtime=0.48 cores=1,3,5,7
/proc/6136/stat : 6136 (python) S 6134 6136 5751 0 -1 4202496 12207 934 1 0 39 9 0 0 20 0 5 0 658253070 327942144 7253 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/6136/statm: 80064 7253 1873 364 0 15973 0
[pid=6158] ppid=6136 vsize=278624 CPUtime=0.01 cores=1,3,5,7
/proc/6158/stat : 6158 (python) S 6136 6136 5751 0 -1 4202560 891 0 0 0 1 0 0 0 18 0 1 0 658253129 285310976 5592 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965263 0 0 16781312 514 14825964326807896326 0 0 17 3 0 0 0
/proc/6158/statm: 69656 5592 377 364 0 5574 0
[pid=6160] ppid=6158 vsize=1017936 CPUtime=2.91 cores=1,3,5,7
/proc/6160/stat : 6160 (gnovelty+2) R 6158 6160 6160 0 -1 4202496 457 0 0 0 291 0 0 0 25 0 1 0 658253129 1042366464 196 33554432000 134512640 135663704 4290297008 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/6160/statm: 254484 196 113 282 0 254200 0
[pid=6161] ppid=6136 vsize=289396 CPUtime=0 cores=1,3,5,7
/proc/6161/stat : 6161 (python) S 6136 6136 5751 0 -1 4202560 912 0 0 0 0 0 0 0 18 0 1 0 658253130 296341504 5729 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965318 0 0 16781312 514 562079694262862086 0 0 17 1 0 0 0
/proc/6161/statm: 72349 5729 370 364 0 8264 0
[pid=6163] ppid=6161 vsize=1017936 CPUtime=5.67 cores=1,3,5,7
/proc/6163/stat : 6163 (gnovelty+2) R 6161 6163 6163 0 -1 4202496 462 0 0 0 567 0 0 0 25 0 1 0 658253130 1042366464 197 33554432000 134512640 135663704 4294649264 18446744073709551615 134545631 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/6163/statm: 254484 197 113 282 0 254200 0
[pid=6164] ppid=6136 vsize=299148 CPUtime=0.01 cores=1,3,5,7
/proc/6164/stat : 6164 (python) S 6136 6136 5751 0 -1 4202560 902 0 0 0 1 0 0 0 15 0 1 0 658253131 306327552 5623 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965318 0 0 16781312 514 10883010076486893831 0 0 17 5 0 0 0
/proc/6164/statm: 74787 5623 371 364 0 10699 0
[pid=6168] ppid=6164 vsize=3908 CPUtime=5.67 cores=1,3,5,7
/proc/6168/stat : 6168 (march_hi) R 6164 6168 6168 0 -1 4202496 994 0 0 0 567 0 0 0 25 0 1 0 658253132 4001792 734 33554432000 4194304 4955518 140734182633792 18446744073709551615 4286687 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/6168/statm: 977 734 104 186 0 790 0
[pid=6166] ppid=6136 vsize=310012 CPUtime=0.01 cores=1,3,5,7
/proc/6166/stat : 6166 (python) S 6136 6136 5751 0 -1 4202560 920 0 0 0 1 0 0 0 15 0 1 0 658253132 317452288 5745 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965318 0 0 16781312 514 3605641679400304901 0 0 17 7 0 0 0
/proc/6166/statm: 77503 5745 371 364 0 13412 0
[pid=6169] ppid=6166 vsize=3408 CPUtime=3.72 cores=1,3,5,7
/proc/6169/stat : 6169 (march_hi) R 6166 6169 6169 0 -1 4202496 899 0 0 0 372 0 0 0 25 0 1 0 658253132 3489792 636 33554432000 4194304 4955518 140734434312144 18446744073709551615 4227679 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/6169/statm: 852 636 104 186 0 665 0
[pid=6136/tid=6159] ppid=6134 vsize=320256 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6159/stat : 6159 (python) S 6134 6136 5751 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 5 0 658253129 327942144 7253 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 7 0 0 0
[pid=6136/tid=6162] ppid=6134 vsize=320256 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6162/stat : 6162 (python) S 6134 6136 5751 0 -1 4202560 23 934 0 0 0 0 0 0 21 0 5 0 658253130 327942144 7253 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 1 0 0 0
[pid=6136/tid=6165] ppid=6134 vsize=320256 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6165/stat : 6165 (python) S 6134 6136 5751 0 -1 4202560 35 934 0 0 0 0 0 0 20 0 5 0 658253131 327942144 7253 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
[pid=6136/tid=6167] ppid=6134 vsize=320256 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6167/stat : 6167 (python) S 6134 6136 5751 0 -1 4202560 18 934 0 0 0 0 0 0 21 0 5 0 658253132 327942144 7253 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 18.48
Current children cumulated vsize (KiB) 3540624

[startup+12.7008 s]
/proc/loadavg: 8.25 8.04 6.14 10/398 6269
/proc/meminfo: memFree=23617272/32951124 swapFree=44485764/67111528
[pid=6136] ppid=6134 vsize=320256 CPUtime=0.48 cores=1,3,5,7
/proc/6136/stat : 6136 (python) S 6134 6136 5751 0 -1 4202496 12207 934 1 0 39 9 0 0 20 0 5 0 658253070 327942144 7253 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/6136/statm: 80064 7253 1873 364 0 15973 0
[pid=6158] ppid=6136 vsize=278624 CPUtime=0.02 cores=1,3,5,7
/proc/6158/stat : 6158 (python) S 6136 6136 5751 0 -1 4202560 895 0 0 0 1 1 0 0 18 0 1 0 658253129 285310976 5592 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965263 0 0 16781312 514 5856333181069787398 0 0 17 3 0 0 0
/proc/6158/statm: 69656 5592 377 364 0 5574 0
[pid=6160] ppid=6158 vsize=1017936 CPUtime=7.11 cores=1,3,5,7
/proc/6160/stat : 6160 (gnovelty+2) R 6158 6160 6160 0 -1 4202496 457 0 0 0 711 0 0 0 25 0 1 0 658253129 1042366464 196 33554432000 134512640 135663704 4290297008 18446744073709551615 134525255 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/6160/statm: 254484 196 113 282 0 254200 0
[pid=6161] ppid=6136 vsize=289396 CPUtime=0.02 cores=1,3,5,7

################
# More data... #
################

[pid=6161/tid=6285] ppid=6136 vsize=289396 CPUtime=0 cores=1,3,5,7
/proc/6161/task/6285/stat : 6285 (python) S 6136 6136 5751 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 658264359 296341504 5760 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 514 18446604457338529920 0 0 -1 1 0 0 0
[pid=6164] ppid=6136 vsize=299148 CPUtime=2.37 cores=1,3,5,7
/proc/6164/stat : 6164 (python) S 6136 6136 5751 0 -1 4202560 1142 0 0 0 130 107 0 0 15 0 2 0 658253131 306327552 5653 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965318 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/6164/statm: 74787 5653 400 364 0 10699 0
[pid=6168] ppid=6164 vsize=176772 CPUtime=792.06 cores=1,3,5,7
/proc/6168/stat : 6168 (march_hi) R 6164 6168 6168 0 -1 4202496 29303 0 0 0 79201 5 0 0 25 0 1 0 658253132 181014528 29043 33554432000 4194304 4955518 140734182633792 18446744073709551615 4238727 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/6168/statm: 44193 29043 104 186 0 44006 0
[pid=6164/tid=6301] ppid=6136 vsize=299148 CPUtime=0 cores=1,3,5,7
/proc/6164/task/6301/stat : 6301 (python) S 6136 6136 5751 0 -1 4202560 44 0 0 0 0 0 0 0 15 0 2 0 658294700 306327552 5653 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 514 18446604468045989760 0 0 -1 5 0 0 0
[pid=6166] ppid=6136 vsize=310012 CPUtime=2.87 cores=1,3,5,7
/proc/6166/stat : 6166 (python) S 6136 6136 5751 0 -1 4202560 1160 0 0 0 151 136 0 0 15 0 2 0 658253132 317452288 5777 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/6166/statm: 77503 5777 400 364 0 13412 0
[pid=6169] ppid=6166 vsize=176772 CPUtime=971.6 cores=1,3,5,7
/proc/6169/stat : 6169 (march_hi) T 6166 6169 6169 0 -1 4202496 35152 0 0 0 97154 6 0 0 25 0 1 0 658253132 181014528 34889 33554432000 4194304 4955518 140734434312144 18446744073709551615 4295634 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/6169/statm: 44193 34889 104 186 0 44006 0
[pid=6166/tid=6312] ppid=6136 vsize=310012 CPUtime=0 cores=1,3,5,7
/proc/6166/task/6312/stat : 6312 (python) S 6136 6136 5751 0 -1 4202560 44 0 0 0 0 0 0 0 15 0 2 0 658322633 317452288 5777 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 514 18446604470730439360 0 0 -1 7 0 0 0
[pid=6282] ppid=6136 vsize=320268 CPUtime=2.32 cores=1,3,5,7
/proc/6282/stat : 6282 (python) S 6136 6136 5751 0 -1 4202560 1152 0 0 0 122 110 0 0 15 0 2 0 658263757 327954432 5797 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965318 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/6282/statm: 80067 5797 400 364 0 15973 0
[pid=6284] ppid=6282 vsize=176772 CPUtime=799.17 cores=1,3,5,7
/proc/6284/stat : 6284 (march_hi) R 6282 6284 6284 0 -1 4202496 29464 0 0 0 79911 6 0 0 25 0 1 0 658263757 181014528 29202 33554432000 4194304 4955518 140733709829392 18446744073709551615 4227664 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/6284/statm: 44193 29202 104 186 0 44006 0
[pid=6282/tid=6327] ppid=6136 vsize=320268 CPUtime=0 cores=1,3,5,7
/proc/6282/task/6327/stat : 6327 (python) S 6136 6136 5751 0 -1 4202560 44 0 0 0 0 0 0 0 16 0 2 0 658332038 327954432 5797 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 514 18446604459387983168 0 0 -1 3 0 0 0
[pid=6286] ppid=6136 vsize=330132 CPUtime=2.54 cores=1,3,5,7
/proc/6286/stat : 6286 (python) S 6136 6136 5751 0 -1 4202560 1151 0 0 0 133 121 0 0 15 0 2 0 658264360 338055168 5719 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965318 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/6286/statm: 82533 5719 400 364 0 18436 0
[pid=6288] ppid=6286 vsize=176772 CPUtime=866.76 cores=1,3,5,7
/proc/6288/stat : 6288 (march_hi) R 6286 6288 6288 0 -1 4202496 31729 0 0 0 86669 7 0 0 25 0 1 0 658264360 181014528 31468 33554432000 4194304 4955518 140733386562768 18446744073709551615 4209688 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/6288/statm: 44193 31468 104 186 0 44006 0
[pid=6286/tid=6329] ppid=6136 vsize=330132 CPUtime=0 cores=1,3,5,7
/proc/6286/task/6329/stat : 6329 (python) S 6136 6136 5751 0 -1 4202560 45 0 0 0 0 0 0 0 17 0 2 0 658332540 338055168 5719 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 514 18446604459387984064 0 0 -1 3 0 0 0
[pid=6302] ppid=6136 vsize=340844 CPUtime=1.98 cores=1,3,5,7
/proc/6302/stat : 6302 (python) S 6136 6136 5751 0 -1 4202560 1155 0 0 0 106 92 0 0 18 0 2 0 658294701 349024256 5821 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965318 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/6302/statm: 85211 5821 400 364 0 21111 0
[pid=6304] ppid=6302 vsize=182580 CPUtime=695.8 cores=1,3,5,7
/proc/6304/stat : 6304 (kcnfs-2006) R 6302 6304 6304 0 -1 4202496 667 0 0 0 69580 0 0 0 25 0 1 0 658294701 186961920 405 33554432000 134512640 135163436 4294361008 18446744073709551615 134564823 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/6304/statm: 45645 405 77 159 0 45485 0
[pid=6302/tid=6337] ppid=6136 vsize=340844 CPUtime=0 cores=1,3,5,7
/proc/6302/task/6337/stat : 6337 (python) S 6136 6136 5751 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 658353961 349024256 5821 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 514 18446604470730437568 0 0 -1 5 0 0 0
[pid=6313] ppid=6136 vsize=350704 CPUtime=1.16 cores=1,3,5,7
/proc/6313/stat : 6313 (python) S 6136 6136 5751 0 -1 4202560 1202 0 0 0 63 53 0 0 16 0 2 0 658322633 359120896 5748 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/6313/statm: 87676 5748 406 364 0 23573 0
[pid=6315] ppid=6313 vsize=82604 CPUtime=391.42 cores=1,3,5,7
/proc/6315/stat : 6315 (march_hi) T 6313 6315 6315 0 -1 4202496 15193 0 0 0 39138 4 0 0 25 0 1 0 658322634 84586496 14923 33554432000 4194304 4955518 140736370547232 18446744073709551615 4224648 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/6315/statm: 20651 14923 104 186 0 20464 0
[pid=6313/tid=6340] ppid=6136 vsize=350704 CPUtime=0 cores=1,3,5,7
/proc/6313/task/6340/stat : 6340 (python) S 6136 6136 5751 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 658361892 359120896 5748 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 514 18446604456510837120 0 0 -1 3 0 0 0
[pid=6136/tid=6159] ppid=6134 vsize=361584 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6159/stat : 6159 (python) S 6134 6136 5751 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 9 0 658253129 370262016 7379 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 7 0 0 0
[pid=6136/tid=6162] ppid=6134 vsize=361584 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6162/stat : 6162 (python) S 6134 6136 5751 0 -1 4202560 23 934 0 0 0 0 0 0 21 0 9 0 658253130 370262016 7379 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 1 0 0 0
[pid=6136/tid=6165] ppid=6134 vsize=361584 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6165/stat : 6165 (python) S 6134 6136 5751 0 -1 4202560 39 934 0 0 0 0 0 0 18 0 9 0 658253131 370262016 7379 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
[pid=6136/tid=6167] ppid=6134 vsize=361584 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6167/stat : 6167 (python) S 6134 6136 5751 0 -1 4202560 24 934 0 0 0 0 0 0 18 0 9 0 658253132 370262016 7379 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
[pid=6136/tid=6283] ppid=6134 vsize=361584 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6283/stat : 6283 (python) S 6134 6136 5751 0 -1 4202560 26 934 0 0 0 0 0 0 18 0 9 0 658263757 370262016 7379 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
[pid=6136/tid=6287] ppid=6134 vsize=361584 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6287/stat : 6287 (python) S 6134 6136 5751 0 -1 4202560 29 934 0 0 0 0 0 0 18 0 9 0 658264360 370262016 7379 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
[pid=6136/tid=6303] ppid=6134 vsize=361584 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6303/stat : 6303 (python) S 6134 6136 5751 0 -1 4202560 29 934 0 0 0 0 0 0 18 0 9 0 658294701 370262016 7379 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
[pid=6136/tid=6314] ppid=6134 vsize=361584 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6314/stat : 6314 (python) S 6134 6136 5751 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 9 0 658322634 370262016 7379 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4731.71
Current children cumulated vsize (KiB) 5899100

[startup+1198.1 s]
/proc/loadavg: 8.22 8.17 7.59 9/439 6343
/proc/meminfo: memFree=22474084/32951124 swapFree=44485764/67111528
[pid=6136] ppid=6134 vsize=361584 CPUtime=0.56 cores=1,3,5,7
/proc/6136/stat : 6136 (python) S 6134 6136 5751 0 -1 4202496 16147 934 1 0 46 10 0 0 18 0 9 0 658253070 370262016 7379 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/6136/statm: 90396 7379 1882 364 0 26293 0
[pid=6158] ppid=6136 vsize=288868 CPUtime=0.28 cores=1,3,5,7
/proc/6158/stat : 6158 (python) S 6136 6136 5751 0 -1 4202560 1165 0 0 0 14 14 0 0 19 0 2 0 658253129 295800832 5636 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/6158/statm: 72217 5636 416 364 0 8135 0
[pid=6160] ppid=6158 vsize=1017936 CPUtime=101.02 cores=1,3,5,7
/proc/6160/stat : 6160 (gnovelty+2) T 6158 6160 6160 0 -1 4202496 457 0 0 0 10102 0 0 0 25 0 1 0 658253129 1042366464 196 33554432000 134512640 135663704 4290297008 18446744073709551615 134524452 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/6160/statm: 254484 196 113 282 0 254200 0
[pid=6158/tid=6281] ppid=6136 vsize=288868 CPUtime=0 cores=1,3,5,7
/proc/6158/task/6281/stat : 6281 (python) S 6136 6136 5751 0 -1 4202560 43 0 0 0 0 0 0 0 19 0 2 0 658263756 295800832 5636 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 514 18446604469249533952 0 0 -1 3 0 0 0
[pid=6161] ppid=6136 vsize=289396 CPUtime=0.29 cores=1,3,5,7
/proc/6161/stat : 6161 (python) S 6136 6136 5751 0 -1 4202560 1101 0 0 0 15 14 0 0 19 0 2 0 658253130 296341504 5760 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/6161/statm: 72349 5760 400 364 0 8264 0
[pid=6163] ppid=6161 vsize=1017936 CPUtime=99.51 cores=1,3,5,7
/proc/6163/stat : 6163 (gnovelty+2) T 6161 6163 6163 0 -1 4202496 462 0 0 0 9951 0 0 0 25 0 1 0 658253130 1042366464 197 33554432000 134512640 135663704 4294649264 18446744073709551615 134524978 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/6163/statm: 254484 197 113 282 0 254200 0
[pid=6161/tid=6285] ppid=6136 vsize=289396 CPUtime=0 cores=1,3,5,7
/proc/6161/task/6285/stat : 6285 (python) S 6136 6136 5751 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 658264359 296341504 5760 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 514 18446604457338529920 0 0 -1 1 0 0 0
[pid=6164] ppid=6136 vsize=299148 CPUtime=2.37 cores=1,3,5,7
/proc/6164/stat : 6164 (python) S 6136 6136 5751 0 -1 4202560 1142 0 0 0 130 107 0 0 15 0 2 0 658253131 306327552 5653 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965318 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/6164/statm: 74787 5653 400 364 0 10699 0
[pid=6168] ppid=6164 vsize=176772 CPUtime=792.25 cores=1,3,5,7
/proc/6168/stat : 6168 (march_hi) R 6164 6168 6168 0 -1 4202496 29310 0 0 0 79220 5 0 0 25 0 1 0 658253132 181014528 29050 33554432000 4194304 4955518 140734182633792 18446744073709551615 4209701 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/6168/statm: 44193 29050 104 186 0 44006 0
[pid=6164/tid=6301] ppid=6136 vsize=299148 CPUtime=0 cores=1,3,5,7
/proc/6164/task/6301/stat : 6301 (python) S 6136 6136 5751 0 -1 4202560 44 0 0 0 0 0 0 0 15 0 2 0 658294700 306327552 5653 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 514 18446604468045989760 0 0 -1 5 0 0 0
[pid=6166] ppid=6136 vsize=310012 CPUtime=2.87 cores=1,3,5,7
/proc/6166/stat : 6166 (python) S 6136 6136 5751 0 -1 4202560 1160 0 0 0 151 136 0 0 15 0 2 0 658253132 317452288 5777 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/6166/statm: 77503 5777 400 364 0 13412 0
[pid=6169] ppid=6166 vsize=176772 CPUtime=971.6 cores=1,3,5,7
/proc/6169/stat : 6169 (march_hi) T 6166 6169 6169 0 -1 4202496 35152 0 0 0 97154 6 0 0 25 0 1 0 658253132 181014528 34889 33554432000 4194304 4955518 140734434312144 18446744073709551615 4295634 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/6169/statm: 44193 34889 104 186 0 44006 0
[pid=6166/tid=6312] ppid=6136 vsize=310012 CPUtime=0 cores=1,3,5,7
/proc/6166/task/6312/stat : 6312 (python) S 6136 6136 5751 0 -1 4202560 44 0 0 0 0 0 0 0 15 0 2 0 658322633 317452288 5777 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 514 18446604470730439360 0 0 -1 7 0 0 0
[pid=6282] ppid=6136 vsize=320268 CPUtime=2.32 cores=1,3,5,7
/proc/6282/stat : 6282 (python) S 6136 6136 5751 0 -1 4202560 1152 0 0 0 122 110 0 0 15 0 2 0 658263757 327954432 5797 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965318 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/6282/statm: 80067 5797 400 364 0 15973 0
[pid=6284] ppid=6282 vsize=176772 CPUtime=799.36 cores=1,3,5,7
/proc/6284/stat : 6284 (march_hi) R 6282 6284 6284 0 -1 4202496 29471 0 0 0 79930 6 0 0 25 0 1 0 658263757 181014528 29209 33554432000 4194304 4955518 140733709829392 18446744073709551615 4227502 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/6284/statm: 44193 29209 104 186 0 44006 0
[pid=6282/tid=6327] ppid=6136 vsize=320268 CPUtime=0 cores=1,3,5,7
/proc/6282/task/6327/stat : 6327 (python) S 6136 6136 5751 0 -1 4202560 44 0 0 0 0 0 0 0 16 0 2 0 658332038 327954432 5797 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 514 18446604459387983168 0 0 -1 3 0 0 0
[pid=6286] ppid=6136 vsize=330132 CPUtime=2.54 cores=1,3,5,7
/proc/6286/stat : 6286 (python) S 6136 6136 5751 0 -1 4202560 1151 0 0 0 133 121 0 0 15 0 2 0 658264360 338055168 5719 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965318 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/6286/statm: 82533 5719 400 364 0 18436 0
[pid=6288] ppid=6286 vsize=176772 CPUtime=866.94 cores=1,3,5,7
/proc/6288/stat : 6288 (march_hi) R 6286 6288 6288 0 -1 4202496 31736 0 0 0 86687 7 0 0 25 0 1 0 658264360 181014528 31475 33554432000 4194304 4955518 140733386562768 18446744073709551615 4209776 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/6288/statm: 44193 31475 104 186 0 44006 0
[pid=6286/tid=6329] ppid=6136 vsize=330132 CPUtime=0 cores=1,3,5,7
/proc/6286/task/6329/stat : 6329 (python) S 6136 6136 5751 0 -1 4202560 45 0 0 0 0 0 0 0 17 0 2 0 658332540 338055168 5719 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 514 18446604459387984064 0 0 -1 3 0 0 0
[pid=6302] ppid=6136 vsize=340844 CPUtime=1.98 cores=1,3,5,7
/proc/6302/stat : 6302 (python) S 6136 6136 5751 0 -1 4202560 1155 0 0 0 106 92 0 0 18 0 2 0 658294701 349024256 5821 33554432000 4194304 5683644 140736125851440 18446744073709551615 244446965318 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/6302/statm: 85211 5821 400 364 0 21111 0
[pid=6304] ppid=6302 vsize=182580 CPUtime=695.99 cores=1,3,5,7
/proc/6304/stat : 6304 (kcnfs-2006) R 6302 6304 6304 0 -1 4202496 667 0 0 0 69599 0 0 0 25 0 1 0 658294701 186961920 405 33554432000 134512640 135163436 4294361008 18446744073709551615 134559503 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/6304/statm: 45645 405 77 159 0 45485 0
[pid=6302/tid=6337] ppid=6136 vsize=340844 CPUtime=0 cores=1,3,5,7
/proc/6302/task/6337/stat : 6337 (python) S 6136 6136 5751 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 658353961 349024256 5821 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 514 18446604470730437568 0 0 -1 5 0 0 0
[pid=6313] ppid=6136 vsize=350704 CPUtime=1.16 cores=1,3,5,7
/proc/6313/stat : 6313 (python) S 6136 6136 5751 0 -1 4202560 1202 0 0 0 63 53 0 0 16 0 2 0 658322633 359120896 5748 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/6313/statm: 87676 5748 406 364 0 23573 0
[pid=6315] ppid=6313 vsize=82604 CPUtime=391.42 cores=1,3,5,7
/proc/6315/stat : 6315 (march_hi) T 6313 6315 6315 0 -1 4202496 15193 0 0 0 39138 4 0 0 25 0 1 0 658322634 84586496 14923 33554432000 4194304 4955518 140736370547232 18446744073709551615 4224648 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/6315/statm: 20651 14923 104 186 0 20464 0
[pid=6313/tid=6340] ppid=6136 vsize=350704 CPUtime=0 cores=1,3,5,7
/proc/6313/task/6340/stat : 6340 (python) S 6136 6136 5751 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 658361892 359120896 5748 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 514 18446604456510837120 0 0 -1 3 0 0 0
[pid=6136/tid=6159] ppid=6134 vsize=361584 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6159/stat : 6159 (python) S 6134 6136 5751 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 9 0 658253129 370262016 7379 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 7 0 0 0
[pid=6136/tid=6162] ppid=6134 vsize=361584 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6162/stat : 6162 (python) S 6134 6136 5751 0 -1 4202560 23 934 0 0 0 0 0 0 21 0 9 0 658253130 370262016 7379 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 1 0 0 0
[pid=6136/tid=6165] ppid=6134 vsize=361584 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6165/stat : 6165 (python) S 6134 6136 5751 0 -1 4202560 39 934 0 0 0 0 0 0 18 0 9 0 658253131 370262016 7379 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
[pid=6136/tid=6167] ppid=6134 vsize=361584 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6167/stat : 6167 (python) S 6134 6136 5751 0 -1 4202560 24 934 0 0 0 0 0 0 18 0 9 0 658253132 370262016 7379 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
[pid=6136/tid=6283] ppid=6134 vsize=361584 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6283/stat : 6283 (python) S 6134 6136 5751 0 -1 4202560 26 934 0 0 0 0 0 0 18 0 9 0 658263757 370262016 7379 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
[pid=6136/tid=6287] ppid=6134 vsize=361584 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6287/stat : 6287 (python) S 6134 6136 5751 0 -1 4202560 29 934 0 0 0 0 0 0 18 0 9 0 658264360 370262016 7379 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
[pid=6136/tid=6303] ppid=6134 vsize=361584 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6303/stat : 6303 (python) S 6134 6136 5751 0 -1 4202560 29 934 0 0 0 0 0 0 18 0 9 0 658294701 370262016 7379 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
[pid=6136/tid=6314] ppid=6134 vsize=361584 CPUtime=0 cores=1,3,5,7
/proc/6136/task/6314/stat : 6314 (python) S 6134 6136 5751 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 9 0 658322634 370262016 7379 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 2 18446604467748333440 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4732.46
Current children cumulated vsize (KiB) 5899100

[startup+1198.2 s]
/proc/loadavg: 8.22 8.17 7.59 9/439 6343
/proc/meminfo: memFree=22474084/32951124 swapFree=44485764/67111528
[pid=6136] ppid=6134 vsize=329764 CPUtime=4732.98 cores=1,3,5,7
/proc/6136/stat : 6136 (python) R 6134 6136 5751 0 -1 4202496 18010 152825 1 0 47 10 472553 688 17 0 3 0 658253070 337678336 7112 33554432000 4194304 5683644 140736125851440 18446744073709551615 5114130 0 0 16781312 0 18446744073709551615 0 0 17 1 0 0 0
/proc/6136/statm: 82441 7112 1865 364 0 18365 0
[pid=6136/tid=6162] ppid=6134 vsize=329764 CPUtime=4732.41 cores=1,3,5,7
/proc/6136/task/6162/stat : 6162 (python) S 6134 6136 5751 0 -1 4202560 26 152825 0 0 0 0 472553 688 18 0 3 0 658253130 337678336 7112 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 0 18446604467748333440 0 0 -1 7 0 0 0
[pid=6136/tid=6283] ppid=6134 vsize=329764 CPUtime=4732.41 cores=1,3,5,7
/proc/6136/task/6283/stat : 6283 (python) S 6134 6136 5751 0 -1 4202560 26 152825 0 0 0 0 472553 688 18 0 3 0 658263757 337678336 7112 33554432000 4194304 5683644 140736125851440 18446744073709551615 244458768817 0 0 16781312 0 18446604467748333440 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4732.98
Current children cumulated vsize (KiB) 329764

Child status: 0
Real time (s): 1198.21
CPU time (s): 4733.01
CPU user time (s): 4726.01
CPU system time (s): 6.99194
CPU usage (%): 395.005
Max. virtual memory (cumulated for all children) (KiB): 5899100

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4726.01
system time used= 6.99194
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 173046
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 8571
involuntary context switches= 27210

runsolver used 5.2542 second user time and 10.6394 second system time

The end

Launcher Data 

Begin job on node105 at 2011-05-15 15:35:26
IDJOB=3378630
IDBENCH=83039
IDSOLVER=1859
FILE ID=node105/3378630-1305466526
RUNJOBID= node105-1305465320-5770
PBS_JOBID= 13325059
Free space on /tmp= 73548 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v700-c2982-S1124205008-074.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378630-1305466526/watcher-3378630-1305466526 -o /tmp/evaluation-result-3378630-1305466526/solver-3378630-1305466526 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378630-1305466526.cnf -seed 108187587 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 7363c9bb5019132082aeb3e4836ce4d1
RANDOM SEED=108187587

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23708188 kB
Buffers:        307884 kB
Cached:        6816404 kB
SwapCached:      39656 kB
Active:        1622520 kB
Inactive:      7336696 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23708188 kB
SwapTotal:    67111528 kB
SwapFree:     44485764 kB
Dirty:           32724 kB
Writeback:           0 kB
AnonPages:     1803248 kB
Mapped:          15960 kB
Slab:           143464 kB
PageTables:      71888 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 91326076 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73516 MiB
End job on node105 at 2011-05-15 15:55:25