Trace number 3385618

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)UNSAT 917.633 169.093

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v450-c1917-S1406921020-013.UNKNOWN.cnf
MD5SUM9869c5a4d8c5bed5ea530735320738db
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark153.377
Satisfiable
(Un)Satisfiability was proved
Number of variables450
Number of clauses1917
Sum of the clauses size5751
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 31917
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.29/0.45	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.29/0.45	c INFO: solving HOME/instance-3385618-1305571099.cnf
0.29/0.45	c INFO: computing task features
0.29/0.45	c INFO: computed variable-clause graph statistics
0.29/0.45	c INFO: computed clause balance statistics
0.29/0.49	c INFO: computed variable balance statistics
0.38/0.51	c INFO: computed small-clause counts
0.38/0.52	c INFO: computed Horn-clause counts
0.38/0.52	c INFO: computed variable graph statistics
0.38/0.52	c INFO: collected features for HOME/instance-3385618-1305571099.cnf in 0.07 s
0.38/0.55	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.56)
0.38/0.58	c INFO: running march_hi@0 for 388 with 4999 remaining (b = 0.37)
0.49/0.61	c INFO: running march_hi@0 for 388 with 4999 remaining (b = 0.33)
0.49/0.64	c INFO: running march_hi@0 for 485 with 4999 remaining (b = 0.33)
0.49/0.67	c INFO: running march_hi@0 for 97 with 4999 remaining (b = 0.09)
0.58/0.70	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.07)
0.58/0.74	c INFO: running march_hi@0 for 485 with 4999 remaining (b = 0.24)
0.58/0.77	c INFO: running march_hi@0 for 485 with 4999 remaining (b = 0.21)
545.28/100.86	c INFO: running march_hi@0 for 485 with 4899 remaining (b = 0.19)
907.30/167.21	c INFO: running kcnfs-2006@0 for 582 with 4833 remaining (b = 0.19)
917.28/169.06	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385618-1305571099/watcher-3385618-1305571099 -o /tmp/evaluation-result-3385618-1305571099/solver-3385618-1305571099 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385618-1305571099.cnf -seed 307021497 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.91 8.76 8.68 5/385 5148
/proc/meminfo: memFree=17514580/32951124 swapFree=67056788/67111528
[pid=5148] ppid=5146 vsize=15776 CPUtime=0 cores=0-7
/proc/5148/stat : 5148 (python) R 5146 5148 3941 0 -1 4202496 883 0 0 0 0 0 0 0 17 0 1 0 668707809 16154624 766 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550198469 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/5148/statm: 3944 767 343 364 0 431 0

[startup+0.0268401 s]
/proc/loadavg: 8.91 8.76 8.68 5/385 5148
/proc/meminfo: memFree=17514580/32951124 swapFree=67056788/67111528
[pid=5148] ppid=5146 vsize=18884 CPUtime=0.01 cores=0-7
/proc/5148/stat : 5148 (python) R 5146 5148 3941 0 -1 4202496 1237 0 0 0 1 0 0 0 17 0 1 0 668707809 19337216 1094 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550535680 0 2147483391 16781312 2 0 0 0 17 2 0 0 0
/proc/5148/statm: 4721 1094 421 364 0 690 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18884

[startup+0.100827 s]
/proc/loadavg: 8.91 8.76 8.68 5/385 5148
/proc/meminfo: memFree=17514580/32951124 swapFree=67056788/67111528
[pid=5148] ppid=5146 vsize=104616 CPUtime=0.06 cores=0-7
/proc/5148/stat : 5148 (python) R 5146 5148 3941 0 -1 4202496 3151 0 0 0 5 1 0 0 17 0 1 0 668707809 107126784 2828 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550535680 0 2147483391 16781312 2 0 0 0 17 2 0 0 0
/proc/5148/statm: 26155 2828 847 364 0 1903 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 104616

[startup+0.300823 s]
/proc/loadavg: 8.91 8.76 8.68 5/385 5148
/proc/meminfo: memFree=17514580/32951124 swapFree=67056788/67111528
[pid=5148] ppid=5146 vsize=198920 CPUtime=0.21 cores=0-7
/proc/5148/stat : 5148 (python) R 5146 5148 3941 0 -1 4202496 7020 935 1 0 16 5 0 0 17 0 1 0 668707809 203694080 5054 33554432000 4194304 5683644 140736755887824 18446744073709551615 47414078892899 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/5148/statm: 49730 5054 1466 364 0 3757 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 198920

[startup+0.70072 s]
/proc/loadavg: 8.91 8.76 8.68 5/385 5148
/proc/meminfo: memFree=17514580/32951124 swapFree=67056788/67111528
[pid=5148] ppid=5146 vsize=395876 CPUtime=0.58 cores=0-7
/proc/5148/stat : 5148 (python) R 5146 5148 3941 0 -1 4202496 13248 935 1 0 51 7 0 0 22 0 6 0 668707809 405377024 7371 33554432000 4194304 5683644 140736755887824 18446744073709551615 271554019661 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/5148/statm: 98969 7371 1882 364 0 34875 0
Current children cumulated CPU time (s) 0.58
Current children cumulated vsize (KiB) 395876

[startup+1.50157 s]
/proc/loadavg: 8.52 8.68 8.66 13/410 5177
/proc/meminfo: memFree=17459184/32951124 swapFree=67056788/67111528
[pid=5148] ppid=5146 vsize=426644 CPUtime=0.65 cores=0-7
/proc/5148/stat : 5148 (python) S 5146 5148 3941 0 -1 4202496 15188 935 1 0 57 8 0 0 25 0 9 0 668707809 436883456 7423 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562364363 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/5148/statm: 106661 7423 1889 364 0 42558 0
[pid=5154] ppid=5148 vsize=278596 CPUtime=0 cores=0-7
/proc/5154/stat : 5154 (python) S 5148 5148 3941 0 -1 4202560 738 0 0 0 0 0 0 0 18 0 1 0 668707864 285282304 5556 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557711 0 0 16781312 514 7919398529607401471 0 0 17 3 0 0 0
/proc/5154/statm: 69649 5556 354 364 0 5567 0
[pid=5156] ppid=5154 vsize=1017800 CPUtime=0.94 cores=0-7
/proc/5156/stat : 5156 (gnovelty+2) R 5154 5156 5156 0 -1 4202496 442 0 0 0 94 0 0 0 22 0 1 0 668707865 1042227200 174 33554432000 134512640 135663704 4287637024 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/5156/statm: 254450 174 113 282 0 254166 0
[pid=5157] ppid=5148 vsize=289328 CPUtime=0 cores=0-7
/proc/5157/stat : 5157 (python) S 5148 5148 3941 0 -1 4202560 743 0 0 0 0 0 0 0 15 0 1 0 668707867 296271872 5689 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 5607385185692647682 0 0 17 3 0 0 0
/proc/5157/statm: 72332 5689 348 364 0 8247 0
[pid=5159] ppid=5157 vsize=1996 CPUtime=0.49 cores=0-7
/proc/5159/stat : 5159 (march_hi) R 5157 5159 5159 0 -1 4202496 589 0 0 0 49 0 0 0 23 0 1 0 668707868 2043904 325 33554432000 4194304 4955518 140734992151088 18446744073709551615 4209688 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/5159/statm: 499 325 104 186 0 312 0
[pid=5160] ppid=5148 vsize=365120 CPUtime=0 cores=0-7
/proc/5160/stat : 5160 (python) S 5148 5148 3941 0 -1 4202560 710 0 0 0 0 0 0 0 15 0 1 0 668707870 373882880 5866 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 3510481147039547651 0 0 17 7 0 0 0
/proc/5160/statm: 91280 5866 348 364 0 27192 0
[pid=5162] ppid=5160 vsize=2128 CPUtime=0.85 cores=0-7
/proc/5162/stat : 5162 (march_hi) R 5160 5162 5162 0 -1 4202496 584 0 0 0 85 0 0 0 23 0 1 0 668707872 2179072 334 33554432000 4194304 4955518 140735762847280 18446744073709551615 4227677 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/5162/statm: 532 334 104 186 0 345 0
[pid=5163] ppid=5148 vsize=375376 CPUtime=0 cores=0-7
/proc/5163/stat : 5163 (python) S 5148 5148 3941 0 -1 4202560 722 0 0 0 0 0 0 0 15 0 1 0 668707873 384385024 5827 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 5607385185692647681 0 0 17 3 0 0 0
/proc/5163/statm: 93844 5827 348 364 0 29753 0
[pid=5167] ppid=5163 vsize=1996 CPUtime=0.42 cores=0-7
/proc/5167/stat : 5167 (march_hi) R 5163 5167 5167 0 -1 4202496 580 0 0 0 42 0 0 0 23 0 1 0 668707877 2043904 325 33554432000 4194304 4955518 140733879141376 18446744073709551615 4224648 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/5167/statm: 499 325 104 186 0 312 0
[pid=5165] ppid=5148 vsize=385632 CPUtime=0 cores=0-7
/proc/5165/stat : 5165 (python) S 5148 5148 3941 0 -1 4202560 730 0 0 0 0 0 0 0 15 0 1 0 668707877 394887168 5849 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 10540160360713781506 0 0 17 1 0 0 0
/proc/5165/statm: 96408 5849 348 364 0 32314 0
[pid=5168] ppid=5165 vsize=1996 CPUtime=0.4 cores=0-7
/proc/5168/stat : 5168 (march_hi) R 5165 5168 5168 0 -1 4202496 579 0 0 0 40 0 0 0 25 0 1 0 668707879 2043904 324 33554432000 4194304 4955518 140735672838960 18446744073709551615 4209692 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/5168/statm: 499 324 104 186 0 312 0
[pid=5169] ppid=5148 vsize=395888 CPUtime=0 cores=0-7
/proc/5169/stat : 5169 (python) S 5148 5148 3941 0 -1 4202560 731 0 0 0 0 0 0 0 23 0 1 0 668707880 405389312 5837 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 6320459308559499263 0 0 17 2 0 0 0
/proc/5169/statm: 98972 5837 348 364 0 34875 0
[pid=5171] ppid=5169 vsize=1017800 CPUtime=0.42 cores=0-7
/proc/5171/stat : 5171 (gnovelty+2) R 5169 5171 5171 0 -1 4202496 431 0 0 0 42 0 0 0 25 0 1 0 668707882 1042227200 173 33554432000 134512640 135663704 4291449120 18446744073709551615 134545576 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/5171/statm: 254450 173 113 282 0 254166 0
[pid=5172] ppid=5148 vsize=406144 CPUtime=0 cores=0-7
/proc/5172/stat : 5172 (python) S 5148 5148 3941 0 -1 4202560 741 0 0 0 0 0 0 0 20 0 1 0 668707883 415891456 5857 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 14017123990997270786 0 0 17 2 0 0 0
/proc/5172/statm: 101536 5857 348 364 0 37436 0
[pid=5174] ppid=5172 vsize=1996 CPUtime=0.42 cores=0-7
/proc/5174/stat : 5174 (march_hi) R 5172 5174 5174 0 -1 4202496 582 0 0 0 42 0 0 0 25 0 1 0 668707886 2043904 325 33554432000 4194304 4955518 140735124568656 18446744073709551615 4209701 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/5174/statm: 499 325 104 186 0 312 0
[pid=5175] ppid=5148 vsize=416400 CPUtime=0 cores=0-7
/proc/5175/stat : 5175 (python) S 5148 5148 3941 0 -1 4202560 776 0 0 0 0 0 0 0 23 0 1 0 668707887 426393600 5884 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 9826871983403401474 0 0 17 2 0 0 0
/proc/5175/statm: 104100 5884 354 364 0 39997 0
[pid=5177] ppid=5175 vsize=1996 CPUtime=0.3 cores=0-7
/proc/5177/stat : 5177 (march_hi) R 5175 5177 5177 0 -1 4202496 579 0 0 0 30 0 0 0 25 0 1 0 668707887 2043904 316 33554432000 4194304 4955518 140737391435120 18446744073709551615 4238449 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/5177/statm: 499 316 104 186 0 312 0
[pid=5148/tid=5155] ppid=5146 vsize=426644 CPUtime=0 cores=0-7
/proc/5148/task/5155/stat : 5155 (python) S 5146 5148 3941 0 -1 4202560 26 935 0 0 0 0 0 0 18 0 9 0 668707865 436883456 7423 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 4 0 0 0
[pid=5148/tid=5158] ppid=5146 vsize=426644 CPUtime=0 cores=0-7
/proc/5148/task/5158/stat : 5158 (python) S 5146 5148 3941 0 -1 4202560 24 935 0 0 0 0 0 0 19 0 9 0 668707868 436883456 7423 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 7 0 0 0
[pid=5148/tid=5161] ppid=5146 vsize=426644 CPUtime=0 cores=0-7
/proc/5148/task/5161/stat : 5161 (python) S 5146 5148 3941 0 -1 4202560 73 935 0 0 0 0 0 0 19 0 9 0 668707871 436883456 7423 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 7 0 0 0
[pid=5148/tid=5164] ppid=5146 vsize=426644 CPUtime=0 cores=0-7
/proc/5148/task/5164/stat : 5164 (python) S 5146 5148 3941 0 -1 4202560 43 935 0 0 0 0 0 0 21 0 9 0 668707874 436883456 7423 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
[pid=5148/tid=5166] ppid=5146 vsize=426644 CPUtime=0 cores=0-7
/proc/5148/task/5166/stat : 5166 (python) S 5146 5148 3941 0 -1 4202560 23 935 0 0 0 0 0 0 22 0 9 0 668707877 436883456 7423 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
[pid=5148/tid=5170] ppid=5146 vsize=426644 CPUtime=0 cores=0-7
/proc/5148/task/5170/stat : 5170 (python) S 5146 5148 3941 0 -1 4202560 32 935 0 0 0 0 0 0 24 0 9 0 668707880 436883456 7423 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
[pid=5148/tid=5173] ppid=5146 vsize=426644 CPUtime=0 cores=0-7
/proc/5148/task/5173/stat : 5173 (python) S 5146 5148 3941 0 -1 4202560 34 935 0 0 0 0 0 0 25 0 9 0 668707883 436883456 7423 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
[pid=5148/tid=5176] ppid=5146 vsize=426644 CPUtime=0 cores=0-7
/proc/5148/task/5176/stat : 5176 (python) S 5146 5148 3941 0 -1 4202560 34 935 0 0 0 0 0 0 25 0 9 0 668707887 436883456 7423 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4.89
Current children cumulated vsize (KiB) 5386836

[startup+3.10128 s]
/proc/loadavg: 8.52 8.68 8.66 13/410 5177
/proc/meminfo: memFree=17458440/32951124 swapFree=67056788/67111528
[pid=5148] ppid=5146 vsize=426644 CPUtime=0.65 cores=0-7
/proc/5148/stat : 5148 (python) S 5146 5148 3941 0 -1 4202496 15188 935 1 0 57 8 0 0 25 0 9 0 668707809 436883456 7423 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562364363 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/5148/statm: 106661 7423 1889 364 0 42558 0
[pid=5154] ppid=5148 vsize=278596 CPUtime=0 cores=0-7
/proc/5154/stat : 5154 (python) S 5148 5148 3941 0 -1 4202560 738 0 0 0 0 0 0 0 18 0 1 0 668707864 285282304 5556 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557711 0 0 16781312 514 7919398529607368962 0 0 17 3 0 0 0
/proc/5154/statm: 69649 5556 354 364 0 5567 0
[pid=5156] ppid=5154 vsize=1017800 CPUtime=2.53 cores=0-7
/proc/5156/stat : 5156 (gnovelty+2) R 5154 5156 5156 0 -1 4202496 442 0 0 0 253 0 0 0 25 0 1 0 668707865 1042227200 174 33554432000 134512640 135663704 4287637024 18446744073709551615 134544196 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/5156/statm: 254450 174 113 282 0 254166 0
[pid=5157] ppid=5148 vsize=289328 CPUtime=0 cores=0-7
/proc/5157/stat : 5157 (python) S 5148 5148 3941 0 -1 4202560 745 0 0 0 0 0 0 0 15 0 1 0 668707867 296271872 5689 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 10540155962667270402 0 0 17 3 0 0 0
/proc/5157/statm: 72332 5689 348 364 0 8247 0
[pid=5159] ppid=5157 vsize=2128 CPUtime=1.28 cores=0-7
/proc/5159/stat : 5159 (march_hi) R 5157 5159 5159 0 -1 4202496 605 0 0 0 128 0 0 0 25 0 1 0 668707868 2179072 341 33554432000 4194304 4955518 140734992151088 18446744073709551615 4209688 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/5159/statm: 532 341 104 186 0 345 0
[pid=5160] ppid=5148 vsize=365120 CPUtime=0 cores=0-7
/proc/5160/stat : 5160 (python) S 5148 5148 3941 0 -1 4202560 712 0 0 0 0 0 0 0 15 0 1 0 668707870 373882880 5866 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 5607385189987614979 0 0 17 7 0 0 0
/proc/5160/statm: 91280 5866 348 364 0 27192 0
[pid=5162] ppid=5160 vsize=2128 CPUtime=2.44 cores=0-7
/proc/5162/stat : 5162 (march_hi) R 5160 5162 5162 0 -1 4202496 601 0 0 0 244 0 0 0 25 0 1 0 668707872 2179072 351 33554432000 4194304 4955518 140735762847280 18446744073709551615 4209737 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/5162/statm: 532 351 104 186 0 345 0
[pid=5163] ppid=5148 vsize=375376 CPUtime=0 cores=0-7
/proc/5163/stat : 5163 (python) S 5148 5148 3941 0 -1 4202560 723 0 0 0 0 0 0 0 15 0 1 0 668707873 384385024 5827 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 5607385189987614977 0 0 17 3 0 0 0
/proc/5163/statm: 93844 5827 348 364 0 29753 0
[pid=5167] ppid=5163 vsize=2128 CPUtime=1.22 cores=0-7
/proc/5167/stat : 5167 (march_hi) R 5163 5167 5167 0 -1 4202496 590 0 0 0 122 0 0 0 25 0 1 0 668707877 2179072 335 33554432000 4194304 4955518 140733879141376 18446744073709551615 4261387 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/5167/statm: 532 335 104 186 0 345 0
[pid=5165] ppid=5148 vsize=385632 CPUtime=0 cores=0-7
/proc/5165/stat : 5165 (python) S 5148 5148 3941 0 -1 4202560 732 0 0 0 0 0 0 0 15 0 1 0 668707877 394887168 5849 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 10540160360713781506 0 0 17 1 0 0 0
/proc/5165/statm: 96408 5849 348 364 0 32314 0
[pid=5168] ppid=5165 vsize=2128 CPUtime=1.2 cores=0-7
/proc/5168/stat : 5168 (march_hi) R 5165 5168 5168 0 -1 4202496 589 0 0 0 120 0 0 0 25 0 1 0 668707879 2179072 334 33554432000 4194304 4955518 140735672838960 18446744073709551615 4209727 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/5168/statm: 532 334 104 186 0 345 0
[pid=5169] ppid=5148 vsize=395888 CPUtime=0 cores=0-7
/proc/5169/stat : 5169 (python) S 5148 5148 3941 0 -1 4202560 731 0 0 0 0 0 0 0 18 0 1 0 668707880 405389312 5837 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 6320459308559466755 0 0 17 2 0 0 0
/proc/5169/statm: 98972 5837 348 364 0 34875 0
[pid=5171] ppid=5169 vsize=1017800 CPUtime=1.22 cores=0-7
/proc/5171/stat : 5171 (gnovelty+2) R 5169 5171 5171 0 -1 4202496 431 0 0 0 122 0 0 0 25 0 1 0 668707882 1042227200 173 33554432000 134512640 135663704 4291449120 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/5171/statm: 254450 173 113 282 0 254166 0
[pid=5172] ppid=5148 vsize=406144 CPUtime=0 cores=0-7
/proc/5172/stat : 5172 (python) S 5148 5148 3941 0 -1 4202560 742 0 0 0 0 0 0 0 15 0 1 0 668707883 415891456 5857 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 14017123990997270786 0 0 17 2 0 0 0
/proc/5172/statm: 101536 5857 348 364 0 37436 0
[pid=5174] ppid=5172 vsize=2128 CPUtime=1.22 cores=0-7
/proc/5174/stat : 5174 (march_hi) R 5172 5174 5174 0 -1 4202496 592 0 0 0 122 0 0 0 25 0 1 0 668707886 2179072 335 33554432000 4194304 4955518 140735124568656 18446744073709551615 4238976 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/5174/statm: 532 335 104 186 0 345 0
[pid=5175] ppid=5148 vsize=416400 CPUtime=0 cores=0-7
/proc/5175/stat : 5175 (python) S 5148 5148 3941 0 -1 4202560 777 0 0 0 0 0 0 0 15 0 1 0 668707887 426393600 5884 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 9826871983403401474 0 0 17 2 0 0 0
/proc/5175/statm: 104100 5884 354 364 0 39997 0
[pid=5177] ppid=5175 vsize=2128 CPUtime=1.1 cores=0-7
/proc/5177/stat : 5177 (march_hi) R 5175 5177 5177 0 -1 4202496 596 0 0 0 110 0 0 0 25 0 1 0 668707887 2179072 333 33554432000 4194304 4955518 140737391435120 18446744073709551615 4209692 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/5177/statm: 532 333 104 186 0 345 0
[pid=5148/tid=5155] ppid=5146 vsize=426644 CPUtime=0 cores=0-7
/proc/5148/task/5155/stat : 5155 (python) S 5146 5148 3941 0 -1 4202560 26 935 0 0 0 0 0 0 18 0 9 0 668707865 436883456 7423 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 4 0 0 0
[pid=5148/tid=5158] ppid=5146 vsize=426644 CPUtime=0 cores=0-7
/proc/5148/task/5158/stat : 5158 (python) S 5146 5148 3941 0 -1 4202560 24 935 0 0 0 0 0 0 19 0 9 0 668707868 436883456 7423 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 7 0 0 0
[pid=5148/tid=5161] ppid=5146 vsize=426644 CPUtime=0 cores=0-7
/proc/5148/task/5161/stat : 5161 (python) S 5146 5148 3941 0 -1 4202560 73 935 0 0 0 0 0 0 19 0 9 0 668707871 436883456 7423 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 7 0 0 0
[pid=5148/tid=5164] ppid=5146 vsize=426644 CPUtime=0 cores=0-7
/proc/5148/task/5164/stat : 5164 (python) S 5146 5148 3941 0 -1 4202560 43 935 0 0 0 0 0 0 21 0 9 0 668707874 436883456 7423 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
[pid=5148/tid=5166] ppid=5146 vsize=426644 CPUtime=0 cores=0-7

################
# More data... #
################

/proc/5148/task/5188/stat : 5188 (python) S 5146 5148 3941 0 -1 4202560 35 935 0 0 0 0 0 0 19 0 11 0 668724531 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 915.62
Current children cumulated vsize (KiB) 6488160

[startup+168.901 s]
/proc/loadavg: 12.06 10.21 9.24 13/418 5189
/proc/meminfo: memFree=17422376/32951124 swapFree=67056788/67111528
[pid=5148] ppid=5146 vsize=447156 CPUtime=0.73 cores=0-7
/proc/5148/stat : 5148 (python) S 5146 5148 3941 0 -1 4202496 16788 935 1 0 64 9 0 0 19 0 11 0 668707809 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562364363 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/5148/statm: 111789 7461 1894 364 0 47680 0
[pid=5154] ppid=5148 vsize=288840 CPUtime=0.28 cores=0-7
/proc/5154/stat : 5154 (python) S 5148 5148 3941 0 -1 4202560 1080 0 0 0 14 14 0 0 19 0 2 0 668707864 295772160 5617 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562364363 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/5154/statm: 72210 5617 410 364 0 8128 0
[pid=5156] ppid=5154 vsize=1017800 CPUtime=99.35 cores=0-7
/proc/5156/stat : 5156 (gnovelty+2) T 5154 5156 5156 0 -1 4202496 442 0 0 0 9935 0 0 0 25 0 1 0 668707865 1042227200 174 33554432000 134512640 135663704 4287637024 18446744073709551615 134546752 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/5156/statm: 254450 174 113 282 0 254166 0
[pid=5154/tid=5180] ppid=5148 vsize=288840 CPUtime=0 cores=0-7
/proc/5154/task/5180/stat : 5180 (python) S 5148 5148 3941 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 668717892 295772160 5617 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 514 18446604446166637312 0 0 -1 2 0 0 0
[pid=5157] ppid=5148 vsize=289328 CPUtime=0.56 cores=0-7
/proc/5157/stat : 5157 (python) S 5148 5148 3941 0 -1 4202560 975 0 0 0 30 26 0 0 15 0 1 0 668707867 296271872 5712 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 10540155962667270402 0 0 17 1 0 0 0
/proc/5157/statm: 72332 5712 371 364 0 8247 0
[pid=5159] ppid=5157 vsize=11008 CPUtime=151.77 cores=0-7
/proc/5159/stat : 5159 (march_hi) R 5157 5159 5159 0 -1 4202496 2235 0 0 0 15177 0 0 0 25 0 1 0 668707868 11272192 1970 33554432000 4194304 4955518 140734992151088 18446744073709551615 4307236 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/5159/statm: 2752 1970 104 186 0 2565 0
[pid=5160] ppid=5148 vsize=365120 CPUtime=0.87 cores=0-7
/proc/5160/stat : 5160 (python) R 5148 5148 3941 0 -1 4202560 809 0 0 0 58 29 0 0 16 0 1 0 668707870 373882880 5889 33554432000 4194304 5683644 140736755887824 18446744073709551615 4952617 0 0 16781312 514 0 0 0 17 7 0 0 0
/proc/5160/statm: 91280 5889 371 364 0 27192 0
[pid=5162] ppid=5160 vsize=0 CPUtime=156.7 cores=0-7
/proc/5162/stat : 5162 (march_hi) R 5160 5162 5162 0 -1 4202500 2291 0 0 0 15670 0 0 0 25 0 1 0 668707872 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 7 0 0 0
/proc/5162/statm: 0 0 0 0 0 0 0
[pid=5163] ppid=5148 vsize=375376 CPUtime=0.53 cores=0-7
/proc/5163/stat : 5163 (python) S 5148 5148 3941 0 -1 4202560 806 0 0 0 29 24 0 0 15 0 1 0 668707873 384385024 5850 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 2339074652946530561 0 0 17 3 0 0 0
/proc/5163/statm: 93844 5850 371 364 0 29753 0
[pid=5167] ppid=5163 vsize=4912 CPUtime=84.1 cores=0-7
/proc/5167/stat : 5167 (march_hi) R 5163 5167 5167 0 -1 4202496 1260 0 0 0 8410 0 0 0 25 0 1 0 668707877 5029888 1004 33554432000 4194304 4955518 140733879141376 18446744073709551615 4227819 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/5167/statm: 1228 1004 104 186 0 1041 0
[pid=5165] ppid=5148 vsize=385632 CPUtime=0.53 cores=0-7
/proc/5165/stat : 5165 (python) S 5148 5148 3941 0 -1 4202560 816 0 0 0 29 24 0 0 15 0 1 0 668707877 394887168 5872 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 10540160360713781506 0 0 17 1 0 0 0
/proc/5165/statm: 96408 5872 371 364 0 32314 0
[pid=5168] ppid=5165 vsize=4912 CPUtime=84.1 cores=0-7
/proc/5168/stat : 5168 (march_hi) R 5165 5168 5168 0 -1 4202496 1257 0 0 0 8410 0 0 0 25 0 1 0 668707879 5029888 1001 33554432000 4194304 4955518 140735672838960 18446744073709551615 4227571 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/5168/statm: 1228 1001 104 186 0 1041 0
[pid=5169] ppid=5148 vsize=395888 CPUtime=0.47 cores=0-7
/proc/5169/stat : 5169 (python) S 5148 5148 3941 0 -1 4202560 978 0 0 0 25 22 0 0 19 0 2 0 668707880 405389312 5890 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562364363 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/5169/statm: 98972 5890 400 364 0 34875 0
[pid=5171] ppid=5169 vsize=1017800 CPUtime=99.11 cores=0-7
/proc/5171/stat : 5171 (gnovelty+2) T 5169 5171 5171 0 -1 4202496 431 0 0 0 9911 0 0 0 25 0 1 0 668707882 1042227200 173 33554432000 134512640 135663704 4291449120 18446744073709551615 134545631 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/5171/statm: 254450 173 113 282 0 254166 0
[pid=5169/tid=5186] ppid=5148 vsize=395888 CPUtime=0 cores=0-7
/proc/5169/task/5186/stat : 5186 (python) S 5148 5148 3941 0 -1 4202560 62 0 0 0 0 0 0 0 19 0 2 0 668724527 405389312 5890 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 514 18446604441499387968 0 0 -1 1 0 0 0
[pid=5172] ppid=5148 vsize=406144 CPUtime=0.54 cores=0-7
/proc/5172/stat : 5172 (python) S 5148 5148 3941 0 -1 4202560 837 0 0 0 29 25 0 0 15 0 1 0 668707883 415891456 5880 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 14017123990997270786 0 0 17 2 0 0 0
/proc/5172/statm: 101536 5880 371 364 0 37436 0
[pid=5174] ppid=5172 vsize=8000 CPUtime=116.59 cores=0-7
/proc/5174/stat : 5174 (march_hi) R 5172 5174 5174 0 -1 4202496 1665 0 0 0 11659 0 0 0 25 0 1 0 668707886 8192000 1407 33554432000 4194304 4955518 140735124568656 18446744073709551615 4227824 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/5174/statm: 2000 1407 104 186 0 1813 0
[pid=5175] ppid=5148 vsize=416400 CPUtime=0.54 cores=0-7
/proc/5175/stat : 5175 (python) S 5148 5148 3941 0 -1 4202560 880 0 0 0 29 25 0 0 15 0 1 0 668707887 426393600 5907 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 9826871983403401474 0 0 17 2 0 0 0
/proc/5175/statm: 104100 5907 377 364 0 39997 0
[pid=5177] ppid=5175 vsize=4912 CPUtime=83.95 cores=0-7
/proc/5177/stat : 5177 (march_hi) R 5175 5177 5177 0 -1 4202496 1265 0 0 0 8395 0 0 0 25 0 1 0 668707887 5029888 1001 33554432000 4194304 4955518 140737391435120 18446744073709551615 4224628 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/5177/statm: 1228 1001 104 186 0 1041 0
[pid=5181] ppid=5148 vsize=426660 CPUtime=0.22 cores=0-7
/proc/5181/stat : 5181 (python) R 5148 5148 3941 0 -1 4202560 850 0 0 0 12 10 0 0 16 0 1 0 668717895 436899840 5911 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550537531 0 0 16781312 514 0 0 0 17 4 0 0 0
/proc/5181/statm: 106664 5911 371 364 0 42558 0
[pid=5183] ppid=5181 vsize=2912 CPUtime=34.83 cores=0-7
/proc/5183/stat : 5183 (march_hi) R 5181 5183 5183 0 -1 4202496 811 0 0 0 3483 0 0 0 25 0 1 0 668717896 2981888 546 33554432000 4194304 4955518 140734724802864 18446744073709551615 4298080 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/5183/statm: 728 546 104 186 0 541 0
[pid=5187] ppid=5148 vsize=436912 CPUtime=0 cores=0-7
/proc/5187/stat : 5187 (python) S 5148 5148 3941 0 -1 4202560 766 0 0 0 0 0 0 0 18 0 1 0 668724531 447397888 5911 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 7567410222940028927 0 0 17 1 0 0 0
/proc/5187/statm: 109228 5911 348 364 0 45119 0
[pid=5189] ppid=5187 vsize=182452 CPUtime=0.81 cores=0-7
/proc/5189/stat : 5189 (kcnfs-2006) R 5187 5189 5189 0 -1 4202496 569 0 0 0 81 0 0 0 23 0 1 0 668724531 186830848 308 33554432000 134512640 135163436 4292486688 18446744073709551615 134569433 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/5189/statm: 45613 308 77 159 0 45453 0
[pid=5148/tid=5155] ppid=5146 vsize=447156 CPUtime=0 cores=0-7
/proc/5148/task/5155/stat : 5155 (python) S 5146 5148 3941 0 -1 4202560 26 935 0 0 0 0 0 0 18 0 11 0 668707865 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 4 0 0 0
[pid=5148/tid=5158] ppid=5146 vsize=447156 CPUtime=0 cores=0-7
/proc/5148/task/5158/stat : 5158 (python) S 5146 5148 3941 0 -1 4202560 24 935 0 0 0 0 0 0 19 0 11 0 668707868 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 7 0 0 0
[pid=5148/tid=5161] ppid=5146 vsize=447156 CPUtime=0 cores=0-7
/proc/5148/task/5161/stat : 5161 (python) S 5146 5148 3941 0 -1 4202560 73 935 0 0 0 0 0 0 19 0 11 0 668707871 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 7 0 0 0
[pid=5148/tid=5164] ppid=5146 vsize=447156 CPUtime=0 cores=0-7
/proc/5148/task/5164/stat : 5164 (python) S 5146 5148 3941 0 -1 4202560 43 935 0 0 0 0 0 0 21 0 11 0 668707874 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
[pid=5148/tid=5166] ppid=5146 vsize=447156 CPUtime=0 cores=0-7
/proc/5148/task/5166/stat : 5166 (python) S 5146 5148 3941 0 -1 4202560 23 935 0 0 0 0 0 0 22 0 11 0 668707877 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
[pid=5148/tid=5170] ppid=5146 vsize=447156 CPUtime=0 cores=0-7
/proc/5148/task/5170/stat : 5170 (python) S 5146 5148 3941 0 -1 4202560 32 935 0 0 0 0 0 0 24 0 11 0 668707880 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
[pid=5148/tid=5173] ppid=5146 vsize=447156 CPUtime=0 cores=0-7
/proc/5148/task/5173/stat : 5173 (python) S 5146 5148 3941 0 -1 4202560 34 935 0 0 0 0 0 0 25 0 11 0 668707883 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
[pid=5148/tid=5176] ppid=5146 vsize=447156 CPUtime=0 cores=0-7
/proc/5148/task/5176/stat : 5176 (python) S 5146 5148 3941 0 -1 4202560 34 935 0 0 0 0 0 0 25 0 11 0 668707887 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
[pid=5148/tid=5182] ppid=5146 vsize=447156 CPUtime=0 cores=0-7
/proc/5148/task/5182/stat : 5182 (python) S 5146 5148 3941 0 -1 4202560 38 935 0 0 0 0 0 0 19 0 11 0 668717896 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 7 0 0 0
[pid=5148/tid=5188] ppid=5146 vsize=447156 CPUtime=0 cores=0-7
/proc/5148/task/5188/stat : 5188 (python) S 5146 5148 3941 0 -1 4202560 35 935 0 0 0 0 0 0 19 0 11 0 668724531 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 916.58
Current children cumulated vsize (KiB) 6488164

[startup+169.036 s]
/proc/loadavg: 12.06 10.21 9.24 13/418 5189
/proc/meminfo: memFree=17422376/32951124 swapFree=67056788/67111528
[pid=5148] ppid=5146 vsize=447156 CPUtime=199.97 cores=0-7
/proc/5148/stat : 5148 (python) S 5146 5148 3941 0 -1 4202496 16829 3916 1 0 64 9 19887 37 17 0 11 0 668707809 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562367807 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/5148/statm: 111789 7461 1894 364 0 47680 0
[pid=5157] ppid=5148 vsize=289328 CPUtime=0.56 cores=0-7
/proc/5157/stat : 5157 (python) S 5148 5148 3941 0 -1 4202560 975 0 0 0 30 26 0 0 15 0 1 0 668707867 296271872 5712 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 10540155962667270402 0 0 17 1 0 0 0
/proc/5157/statm: 72332 5712 371 364 0 8247 0
[pid=5159] ppid=5157 vsize=11008 CPUtime=151.9 cores=0-7
/proc/5159/stat : 5159 (march_hi) R 5157 5159 5159 0 -1 4202496 2237 0 0 0 15190 0 0 0 25 0 1 0 668707868 11272192 1972 33554432000 4194304 4955518 140734992151088 18446744073709551615 4227593 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/5159/statm: 2752 1972 104 186 0 2565 0
[pid=5160] ppid=5148 vsize=0 CPUtime=157.66 cores=0-7
/proc/5160/stat : 5160 (python) Z 5148 5148 3941 0 -1 4202588 1106 2291 0 0 66 30 15670 0 17 0 1 0 668707870 0 0 33554432000 0 0 0 0 0 0 0 16781824 2 18446744073709551615 0 0 17 7 0 0 0
/proc/5160/statm: 0 0 0 0 0 0 0
[pid=5163] ppid=5148 vsize=375376 CPUtime=0.53 cores=0-7
/proc/5163/stat : 5163 (python) S 5148 5148 3941 0 -1 4202560 806 0 0 0 29 24 0 0 15 0 1 0 668707873 384385024 5850 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 2339074652946530561 0 0 17 3 0 0 0
/proc/5163/statm: 93844 5850 371 364 0 29753 0
[pid=5167] ppid=5163 vsize=4912 CPUtime=84.17 cores=0-7
/proc/5167/stat : 5167 (march_hi) R 5163 5167 5167 0 -1 4202496 1261 0 0 0 8417 0 0 0 25 0 1 0 668707877 5029888 1005 33554432000 4194304 4955518 140733879141376 18446744073709551615 4261466 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/5167/statm: 1228 1005 104 186 0 1041 0
[pid=5165] ppid=5148 vsize=385632 CPUtime=0.53 cores=0-7
/proc/5165/stat : 5165 (python) S 5148 5148 3941 0 -1 4202560 816 0 0 0 29 24 0 0 15 0 1 0 668707877 394887168 5872 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 10540160360713781506 0 0 17 1 0 0 0
/proc/5165/statm: 96408 5872 371 364 0 32314 0
[pid=5168] ppid=5165 vsize=4912 CPUtime=84.15 cores=0-7
/proc/5168/stat : 5168 (march_hi) R 5165 5168 5168 0 -1 4202496 1257 0 0 0 8415 0 0 0 25 0 1 0 668707879 5029888 1001 33554432000 4194304 4955518 140735672838960 18446744073709551615 4286872 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/5168/statm: 1228 1001 104 186 0 1041 0
[pid=5172] ppid=5148 vsize=406144 CPUtime=0.54 cores=0-7
/proc/5172/stat : 5172 (python) S 5148 5148 3941 0 -1 4202560 837 0 0 0 29 25 0 0 15 0 1 0 668707883 415891456 5880 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 14017123990997270786 0 0 17 2 0 0 0
/proc/5172/statm: 101536 5880 371 364 0 37436 0
[pid=5174] ppid=5172 vsize=8000 CPUtime=116.66 cores=0-7
/proc/5174/stat : 5174 (march_hi) R 5172 5174 5174 0 -1 4202496 1666 0 0 0 11666 0 0 0 25 0 1 0 668707886 8192000 1408 33554432000 4194304 4955518 140735124568656 18446744073709551615 4290147 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/5174/statm: 2000 1408 104 186 0 1813 0
[pid=5175] ppid=5148 vsize=416400 CPUtime=0.54 cores=0-7
/proc/5175/stat : 5175 (python) S 5148 5148 3941 0 -1 4202560 897 0 0 0 29 25 0 0 15 0 1 0 668707887 426393600 5912 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562367807 0 0 16781824 2 18446744071562232103 0 0 17 2 0 0 0
/proc/5175/statm: 104100 5912 382 364 0 39997 0
[pid=5177] ppid=5175 vsize=4912 CPUtime=84 cores=0-7
/proc/5177/stat : 5177 (march_hi) R 5175 5177 5177 0 -1 4202496 1265 0 0 0 8400 0 0 0 25 0 1 0 668707887 5029888 1001 33554432000 4194304 4955518 140737391435120 18446744073709551615 4209727 256 0 16781312 0 0 0 0 17 2 0 0 0
/proc/5177/statm: 1228 1001 104 186 0 1041 0
[pid=5181] ppid=5148 vsize=426656 CPUtime=0.22 cores=0-7
/proc/5181/stat : 5181 (python) S 5148 5148 3941 0 -1 4202560 850 0 0 0 12 10 0 0 16 0 1 0 668717895 436895744 5911 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 9063740540687712516 0 0 17 4 0 0 0
/proc/5181/statm: 106664 5911 371 364 0 42558 0
[pid=5183] ppid=5181 vsize=2912 CPUtime=34.96 cores=0-7
/proc/5183/stat : 5183 (march_hi) R 5181 5183 5183 0 -1 4202496 812 0 0 0 3496 0 0 0 25 0 1 0 668717896 2981888 547 33554432000 4194304 4955518 140734724802864 18446744073709551615 4314812 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/5183/statm: 728 547 104 186 0 541 0
[pid=5187] ppid=5148 vsize=436912 CPUtime=0 cores=0-7
/proc/5187/stat : 5187 (python) S 5148 5148 3941 0 -1 4202560 766 0 0 0 0 0 0 0 18 0 1 0 668724531 447397888 5911 33554432000 4194304 5683644 140736755887824 18446744073709551615 271550557766 0 0 16781312 514 7567410222940028927 0 0 17 1 0 0 0
/proc/5187/statm: 109228 5911 348 364 0 45119 0
[pid=5189] ppid=5187 vsize=182452 CPUtime=0.89 cores=0-7
/proc/5189/stat : 5189 (kcnfs-2006) R 5187 5189 5189 0 -1 4202496 569 0 0 0 89 0 0 0 23 0 1 0 668724531 186830848 308 33554432000 134512640 135163436 4292486688 18446744073709551615 134559211 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/5189/statm: 45613 308 77 159 0 45453 0
[pid=5148/tid=5155] ppid=5146 vsize=447156 CPUtime=199.24 cores=0-7
/proc/5148/task/5155/stat : 5155 (python) S 5146 5148 3941 0 -1 4202560 26 3916 0 0 0 0 19887 37 18 0 11 0 668707865 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 4 0 0 0
[pid=5148/tid=5158] ppid=5146 vsize=447156 CPUtime=199.24 cores=0-7
/proc/5148/task/5158/stat : 5158 (python) S 5146 5148 3941 0 -1 4202560 24 3916 0 0 0 0 19887 37 19 0 11 0 668707868 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 7 0 0 0
[pid=5148/tid=5161] ppid=5146 vsize=447156 CPUtime=199.24 cores=0-7
/proc/5148/task/5161/stat : 5161 (python) S 5146 5148 3941 0 -1 4202560 73 3916 0 0 0 0 19887 37 19 0 11 0 668707871 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 7 0 0 0
[pid=5148/tid=5164] ppid=5146 vsize=447156 CPUtime=199.24 cores=0-7
/proc/5148/task/5164/stat : 5164 (python) S 5146 5148 3941 0 -1 4202560 43 3916 0 0 0 0 19887 37 21 0 11 0 668707874 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
[pid=5148/tid=5166] ppid=5146 vsize=447156 CPUtime=199.24 cores=0-7
/proc/5148/task/5166/stat : 5166 (python) S 5146 5148 3941 0 -1 4202560 23 3916 0 0 0 0 19887 37 22 0 11 0 668707877 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
[pid=5148/tid=5170] ppid=5146 vsize=447156 CPUtime=199.24 cores=0-7
/proc/5148/task/5170/stat : 5170 (python) S 5146 5148 3941 0 -1 4202560 32 3916 0 0 0 0 19887 37 24 0 11 0 668707880 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
[pid=5148/tid=5173] ppid=5146 vsize=447156 CPUtime=199.24 cores=0-7
/proc/5148/task/5173/stat : 5173 (python) S 5146 5148 3941 0 -1 4202560 34 3916 0 0 0 0 19887 37 25 0 11 0 668707883 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
[pid=5148/tid=5176] ppid=5146 vsize=447156 CPUtime=199.24 cores=0-7
/proc/5148/task/5176/stat : 5176 (python) S 5146 5148 3941 0 -1 4202560 34 3916 0 0 0 0 19887 37 25 0 11 0 668707887 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
[pid=5148/tid=5182] ppid=5146 vsize=447156 CPUtime=199.24 cores=0-7
/proc/5148/task/5182/stat : 5182 (python) S 5146 5148 3941 0 -1 4202560 38 3916 0 0 0 0 19887 37 19 0 11 0 668717896 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 7 0 0 0
[pid=5148/tid=5188] ppid=5146 vsize=447156 CPUtime=199.24 cores=0-7
/proc/5148/task/5188/stat : 5188 (python) S 5146 5148 3941 0 -1 4202560 35 3916 0 0 0 0 19887 37 19 0 11 0 668724531 457887744 7461 33554432000 4194304 5683644 140736755887824 18446744073709551615 271562361265 0 0 16781312 2 18446604453934966592 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 917.28
Current children cumulated vsize (KiB) 3402712

Child status: 0
Real time (s): 169.093
CPU time (s): 917.633
CPU user time (s): 915.434
CPU system time (s): 2.19867
CPU usage (%): 542.68
Max. virtual memory (cumulated for all children) (KiB): 6499172

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 915.434
system time used= 2.19867
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 43865
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4887
involuntary context switches= 18697

runsolver used 0.694894 second user time and 1.50377 second system time

The end

Launcher Data 

Begin job on node133 at 2011-05-16 20:38:19
IDJOB=3385618
IDBENCH=82883
IDSOLVER=1870
FILE ID=node133/3385618-1305571099
RUNJOBID= node133-1305564962-3955
PBS_JOBID= 13350117
Free space on /tmp= 73400 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v450-c1917-S1406921020-013.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385618-1305571099/watcher-3385618-1305571099 -o /tmp/evaluation-result-3385618-1305571099/solver-3385618-1305571099 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385618-1305571099.cnf -seed 307021497 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 9869c5a4d8c5bed5ea530735320738db
RANDOM SEED=307021497

node133.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 6741.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      17514860 kB
Buffers:        322804 kB
Cached:        4537960 kB
SwapCached:       7676 kB
Active:       11808616 kB
Inactive:      3310008 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      17514860 kB
SwapTotal:    67111528 kB
SwapFree:     67056788 kB
Dirty:           32948 kB
Writeback:         184 kB
AnonPages:    10256996 kB
Mapped:          31840 kB
Slab:           203360 kB
PageTables:      44116 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 84928844 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73372 MiB
End job on node133 at 2011-05-16 20:41:08