Trace number 3378405

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? 4779.44 1199.14

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v550-c2343-S1694067460-072.UNKNOWN.cnf
MD5SUMf2d2b125797e776fb2e7099729714d99
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables550
Number of clauses2343
Sum of the clauses size7029
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 32343
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.38/0.51	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.38/0.51	c INFO: solving HOME/instance-3378405-1305465323.cnf
0.38/0.51	c INFO: computing task features
0.38/0.52	c INFO: computed variable-clause graph statistics
0.38/0.52	c INFO: computed clause balance statistics
0.38/0.56	c INFO: computed variable balance statistics
0.38/0.60	c INFO: computed small-clause counts
0.38/0.60	c INFO: computed Horn-clause counts
0.47/0.60	c INFO: computed variable graph statistics
0.47/0.60	c INFO: collected features for HOME/instance-3378405-1305465323.cnf in 0.09 s
0.47/0.61	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.57)
0.47/0.62	c INFO: running march_hi@0 for 388 with 1199 remaining (b = 0.33)
0.47/0.63	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.06)
0.47/0.64	c INFO: running march_hi@0 for 388 with 1199 remaining (b = 0.31)
396.22/99.82	c INFO: running march_hi@0 for 679 with 1100 remaining (b = 0.38)
396.22/99.84	c INFO: running march_hi@0 for 679 with 1100 remaining (b = 0.34)
1571.50/394.34	c INFO: running march_hi@0 for 388 with 806 remaining (b = 0.23)
1583.42/397.39	c INFO: running kcnfs-2006@0 for 582 with 803 remaining (b = 0.25)
3129.05/784.62	c INFO: running march_hi@0 for 415 with 415 remaining (b = 0.17)
3139.02/787.16	c INFO: running march_hi@392 for 97 with 413 remaining (b = 0.05)
3147.38/789.24	c INFO: running march_hi@393 for 97 with 411 remaining (b = 0.04)
3523.20/883.66	c INFO: running march_hi@1076 for 316 with 316 remaining (b = 0.19)
3531.14/885.67	c INFO: running march_hi@0 for 314 with 314 remaining (b = 0.12)
3921.28/983.64	c INFO: running march_hi@489 for 216 with 216 remaining (b = 0.09)
4779.28/1199.11	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378405-1305465323/watcher-3378405-1305465323 -o /tmp/evaluation-result-3378405-1305465323/solver-3378405-1305465323 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378405-1305465323.cnf -seed 1051148951 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.16 0.04 0.68 2/275 7698
/proc/meminfo: memFree=20086084/32950904 swapFree=67057400/67111528
[pid=7698] ppid=7696 vsize=15780 CPUtime=0 cores=1,3,5,7
/proc/7698/stat : 7698 (python) R 7696 7698 7577 0 -1 4202496 894 0 0 0 0 0 0 0 22 0 1 0 658132098 16158720 777 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278877696 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/7698/statm: 3945 777 343 364 0 432 0

[startup+0.0979539 s]
/proc/loadavg: 0.16 0.04 0.68 2/275 7698
/proc/meminfo: memFree=20086084/32950904 swapFree=67057400/67111528
[pid=7698] ppid=7696 vsize=99376 CPUtime=0.06 cores=1,3,5,7
/proc/7698/stat : 7698 (python) R 7696 7698 7577 0 -1 4202496 2869 0 0 0 5 1 0 0 17 0 1 0 658132098 101761024 2567 33554432000 4194304 5683644 140733720802656 18446744073709551615 4580080 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/7698/statm: 24844 2567 818 364 0 1695 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 99376

[startup+0.100951 s]
/proc/loadavg: 0.16 0.04 0.68 2/275 7698
/proc/meminfo: memFree=20086084/32950904 swapFree=67057400/67111528
[pid=7698] ppid=7696 vsize=101880 CPUtime=0.06 cores=1,3,5,7
/proc/7698/stat : 7698 (python) R 7696 7698 7577 0 -1 4202496 2955 0 0 0 5 1 0 0 17 0 1 0 658132098 104325120 2650 33554432000 4194304 5683644 140733720802656 18446744073709551615 4622252 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/7698/statm: 25470 2650 837 364 0 1770 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 101880

[startup+0.301043 s]
/proc/loadavg: 0.16 0.04 0.68 2/275 7698
/proc/meminfo: memFree=20086084/32950904 swapFree=67057400/67111528
[pid=7698] ppid=7696 vsize=135900 CPUtime=0.22 cores=1,3,5,7
/proc/7698/stat : 7698 (python) S 7696 7698 7577 0 -1 4202496 6049 933 1 0 16 6 0 0 17 0 1 0 658132098 139161600 4190 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278877696 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/7698/statm: 33975 4190 1032 364 0 3161 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 135900

[startup+0.700846 s]
/proc/loadavg: 0.16 0.04 0.68 2/275 7698
/proc/meminfo: memFree=20086084/32950904 swapFree=67057400/67111528
[pid=7698] ppid=7696 vsize=320268 CPUtime=0.51 cores=1,3,5,7
/proc/7698/stat : 7698 (python) S 7696 7698 7577 0 -1 4202496 12206 933 1 0 40 11 0 0 25 0 5 0 658132098 327954432 7255 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7698/statm: 80067 7255 1873 364 0 15976 0
Current children cumulated CPU time (s) 0.51
Current children cumulated vsize (KiB) 320268

[startup+1.5007 s]
/proc/loadavg: 0.16 0.04 0.68 9/305 7736
/proc/meminfo: memFree=20008220/32950904 swapFree=67057400/67111528
[pid=7698] ppid=7696 vsize=320268 CPUtime=0.51 cores=1,3,5,7
/proc/7698/stat : 7698 (python) S 7696 7698 7577 0 -1 4202496 12206 933 1 0 40 11 0 0 25 0 5 0 658132098 327954432 7255 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7698/statm: 80067 7255 1873 364 0 15976 0
[pid=7713] ppid=7698 vsize=278476 CPUtime=0 cores=1,3,5,7
/proc/7713/stat : 7713 (python) S 7698 7698 7577 0 -1 4202560 728 0 0 0 0 0 0 0 18 0 1 0 658132159 285159424 5552 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899727 0 0 16781312 514 12152799496643805183 0 0 17 5 0 0 0
/proc/7713/statm: 69619 5552 354 364 0 5537 0
[pid=7715] ppid=7713 vsize=1017800 CPUtime=0.88 cores=1,3,5,7
/proc/7715/stat : 7715 (gnovelty+2) R 7713 7715 7715 0 -1 4202496 444 0 0 0 88 0 0 0 21 0 1 0 658132160 1042227200 181 33554432000 134512640 135663704 4293956256 18446744073709551615 134524681 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7715/statm: 254450 181 113 282 0 254166 0
[pid=7716] ppid=7698 vsize=289344 CPUtime=0 cores=1,3,5,7
/proc/7716/stat : 7716 (python) S 7698 7698 7577 0 -1 4202560 748 0 0 0 0 0 0 0 15 0 1 0 658132160 296288256 5693 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899782 0 0 16781312 514 13147128818449744133 0 0 17 7 0 0 0
/proc/7716/statm: 72336 5693 348 364 0 8251 0
[pid=7718] ppid=7716 vsize=2500 CPUtime=0.87 cores=1,3,5,7
/proc/7718/stat : 7718 (march_hi) R 7716 7718 7718 0 -1 4202496 702 0 0 0 87 0 0 0 25 0 1 0 658132161 2560000 437 33554432000 4194304 4955518 140735597678256 18446744073709551615 4238368 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/7718/statm: 625 437 104 186 0 438 0
[pid=7719] ppid=7698 vsize=299228 CPUtime=0 cores=1,3,5,7
/proc/7719/stat : 7719 (python) S 7698 7698 7577 0 -1 4202560 737 0 0 0 0 0 0 0 22 0 1 0 658132162 306409472 5587 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899782 0 0 16781312 514 6895600097153777663 0 0 17 5 0 0 0
/proc/7719/statm: 74807 5587 348 364 0 10719 0
[pid=7721] ppid=7719 vsize=1017800 CPUtime=0.84 cores=1,3,5,7
/proc/7721/stat : 7721 (gnovelty+2) R 7719 7721 7721 0 -1 4202496 444 0 0 0 84 0 0 0 25 0 1 0 658132162 1042227200 182 33554432000 134512640 135663704 4287591280 18446744073709551615 134524697 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/7721/statm: 254450 182 113 282 0 254166 0
[pid=7722] ppid=7698 vsize=310024 CPUtime=0 cores=1,3,5,7
/proc/7722/stat : 7722 (python) S 7698 7698 7577 0 -1 4202560 759 0 0 0 0 0 0 0 18 0 1 0 658132163 317464576 5726 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899782 0 0 16781312 514 16686808591981576456 0 0 17 5 0 0 0
/proc/7722/statm: 77506 5726 348 364 0 13415 0
[pid=7724] ppid=7722 vsize=2500 CPUtime=0.84 cores=1,3,5,7
/proc/7724/stat : 7724 (march_hi) R 7722 7724 7724 0 -1 4202496 698 0 0 0 84 0 0 0 25 0 1 0 658132163 2560000 436 33554432000 4194304 4955518 140737408834880 18446744073709551615 4238696 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/7724/statm: 625 436 104 186 0 438 0
[pid=7698/tid=7714] ppid=7696 vsize=320268 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7714/stat : 7714 (python) S 7696 7698 7577 0 -1 4202560 25 933 0 0 0 0 0 0 19 0 5 0 658132160 327954432 7255 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 5 0 0 0
[pid=7698/tid=7717] ppid=7696 vsize=320268 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7717/stat : 7717 (python) S 7696 7698 7577 0 -1 4202560 24 933 0 0 0 0 0 0 21 0 5 0 658132161 327954432 7255 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=7720] ppid=7696 vsize=320268 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7720/stat : 7720 (python) S 7696 7698 7577 0 -1 4202560 23 933 0 0 0 0 0 0 23 0 5 0 658132162 327954432 7255 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=7723] ppid=7696 vsize=320268 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7723/stat : 7723 (python) S 7696 7698 7577 0 -1 4202560 28 933 0 0 0 0 0 0 25 0 5 0 658132163 327954432 7255 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3.94
Current children cumulated vsize (KiB) 3537940

[startup+3.10142 s]
/proc/loadavg: 0.87 0.19 0.72 6/293 7768
/proc/meminfo: memFree=20022552/32950904 swapFree=67057400/67111528
[pid=7698] ppid=7696 vsize=320268 CPUtime=0.51 cores=1,3,5,7
/proc/7698/stat : 7698 (python) S 7696 7698 7577 0 -1 4202496 12206 933 1 0 40 11 0 0 25 0 5 0 658132098 327954432 7255 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7698/statm: 80067 7255 1873 364 0 15976 0
[pid=7713] ppid=7698 vsize=278476 CPUtime=0 cores=1,3,5,7
/proc/7713/stat : 7713 (python) S 7698 7698 7577 0 -1 4202560 728 0 0 0 0 0 0 0 18 0 1 0 658132159 285159424 5552 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899727 0 0 16781312 514 12152799496643772680 0 0 17 5 0 0 0
/proc/7713/statm: 69619 5552 354 364 0 5537 0
[pid=7715] ppid=7713 vsize=1017800 CPUtime=2.48 cores=1,3,5,7
/proc/7715/stat : 7715 (gnovelty+2) R 7713 7715 7715 0 -1 4202496 444 0 0 0 248 0 0 0 25 0 1 0 658132160 1042227200 181 33554432000 134512640 135663704 4293956256 18446744073709551615 134564723 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7715/statm: 254450 181 113 282 0 254166 0
[pid=7716] ppid=7698 vsize=289344 CPUtime=0 cores=1,3,5,7
/proc/7716/stat : 7716 (python) S 7698 7698 7577 0 -1 4202560 750 0 0 0 0 0 0 0 15 0 1 0 658132160 296288256 5693 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899782 0 0 16781312 514 13147128818449744133 0 0 17 7 0 0 0
/proc/7716/statm: 72336 5693 348 364 0 8251 0
[pid=7718] ppid=7716 vsize=2820 CPUtime=2.47 cores=1,3,5,7
/proc/7718/stat : 7718 (march_hi) R 7716 7718 7718 0 -1 4202496 759 0 0 0 247 0 0 0 25 0 1 0 658132161 2887680 494 33554432000 4194304 4955518 140735597678256 18446744073709551615 4314139 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/7718/statm: 705 494 104 186 0 518 0
[pid=7719] ppid=7698 vsize=299228 CPUtime=0 cores=1,3,5,7
/proc/7719/stat : 7719 (python) S 7698 7698 7577 0 -1 4202560 737 0 0 0 0 0 0 0 18 0 1 0 658132162 306409472 5587 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899782 0 0 16781312 514 6895600097153745157 0 0 17 5 0 0 0
/proc/7719/statm: 74807 5587 348 364 0 10719 0
[pid=7721] ppid=7719 vsize=1017800 CPUtime=2.43 cores=1,3,5,7
/proc/7721/stat : 7721 (gnovelty+2) R 7719 7721 7721 0 -1 4202496 444 0 0 0 243 0 0 0 25 0 1 0 658132162 1042227200 182 33554432000 134512640 135663704 4287591280 18446744073709551615 134525347 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/7721/statm: 254450 182 113 282 0 254166 0
[pid=7722] ppid=7698 vsize=310024 CPUtime=0 cores=1,3,5,7
/proc/7722/stat : 7722 (python) S 7698 7698 7577 0 -1 4202560 761 0 0 0 0 0 0 0 15 0 1 0 658132163 317464576 5726 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899782 0 0 16781312 514 16686808591981576456 0 0 17 5 0 0 0
/proc/7722/statm: 77506 5726 348 364 0 13415 0
[pid=7724] ppid=7722 vsize=2820 CPUtime=2.44 cores=1,3,5,7
/proc/7724/stat : 7724 (march_hi) R 7722 7724 7724 0 -1 4202496 755 0 0 0 244 0 0 0 25 0 1 0 658132163 2887680 493 33554432000 4194304 4955518 140737408834880 18446744073709551615 4206520 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/7724/statm: 705 493 104 186 0 518 0
[pid=7698/tid=7714] ppid=7696 vsize=320268 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7714/stat : 7714 (python) S 7696 7698 7577 0 -1 4202560 25 933 0 0 0 0 0 0 19 0 5 0 658132160 327954432 7255 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 5 0 0 0
[pid=7698/tid=7717] ppid=7696 vsize=320268 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7717/stat : 7717 (python) S 7696 7698 7577 0 -1 4202560 24 933 0 0 0 0 0 0 21 0 5 0 658132161 327954432 7255 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=7720] ppid=7696 vsize=320268 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7720/stat : 7720 (python) S 7696 7698 7577 0 -1 4202560 23 933 0 0 0 0 0 0 23 0 5 0 658132162 327954432 7255 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=7723] ppid=7696 vsize=320268 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7723/stat : 7723 (python) S 7696 7698 7577 0 -1 4202560 28 933 0 0 0 0 0 0 25 0 5 0 658132163 327954432 7255 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 10.33
Current children cumulated vsize (KiB) 3538580

[startup+6.30085 s]
/proc/loadavg: 0.87 0.19 0.72 10/305 7824
/proc/meminfo: memFree=19998388/32950904 swapFree=67057400/67111528
[pid=7698] ppid=7696 vsize=320268 CPUtime=0.51 cores=1,3,5,7
/proc/7698/stat : 7698 (python) S 7696 7698 7577 0 -1 4202496 12206 933 1 0 40 11 0 0 25 0 5 0 658132098 327954432 7255 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7698/statm: 80067 7255 1873 364 0 15976 0
[pid=7713] ppid=7698 vsize=278476 CPUtime=0 cores=1,3,5,7
/proc/7713/stat : 7713 (python) S 7698 7698 7577 0 -1 4202560 889 0 0 0 0 0 0 0 18 0 1 0 658132159 285159424 5575 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899727 0 0 16781312 514 12152799496643772680 0 0 17 5 0 0 0
/proc/7713/statm: 69619 5575 377 364 0 5537 0
[pid=7715] ppid=7713 vsize=1017800 CPUtime=5.67 cores=1,3,5,7
/proc/7715/stat : 7715 (gnovelty+2) R 7713 7715 7715 0 -1 4202496 444 0 0 0 567 0 0 0 25 0 1 0 658132160 1042227200 181 33554432000 134512640 135663704 4293956256 18446744073709551615 134545534 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7715/statm: 254450 181 113 282 0 254166 0
[pid=7716] ppid=7698 vsize=289344 CPUtime=0 cores=1,3,5,7
/proc/7716/stat : 7716 (python) S 7698 7698 7577 0 -1 4202560 924 0 0 0 0 0 0 0 15 0 1 0 658132160 296288256 5716 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899782 0 0 16781312 514 13147128818449744133 0 0 17 7 0 0 0
/proc/7716/statm: 72336 5716 371 364 0 8251 0
[pid=7718] ppid=7716 vsize=3808 CPUtime=5.67 cores=1,3,5,7
/proc/7718/stat : 7718 (march_hi) R 7716 7718 7718 0 -1 4202496 915 0 0 0 567 0 0 0 25 0 1 0 658132161 3899392 650 33554432000 4194304 4955518 140735597678256 18446744073709551615 4224658 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/7718/statm: 952 650 104 186 0 765 0
[pid=7719] ppid=7698 vsize=299228 CPUtime=0 cores=1,3,5,7
/proc/7719/stat : 7719 (python) S 7698 7698 7577 0 -1 4202560 898 0 0 0 0 0 0 0 18 0 1 0 658132162 306409472 5610 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899782 0 0 16781312 514 6895600097153745157 0 0 17 5 0 0 0
/proc/7719/statm: 74807 5610 371 364 0 10719 0
[pid=7721] ppid=7719 vsize=1017800 CPUtime=5.59 cores=1,3,5,7
/proc/7721/stat : 7721 (gnovelty+2) R 7719 7721 7721 0 -1 4202496 444 0 0 0 559 0 0 0 25 0 1 0 658132162 1042227200 182 33554432000 134512640 135663704 4287591280 18446744073709551615 134524670 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/7721/statm: 254450 182 113 282 0 254166 0
[pid=7722] ppid=7698 vsize=310024 CPUtime=0 cores=1,3,5,7
/proc/7722/stat : 7722 (python) S 7698 7698 7577 0 -1 4202560 922 0 0 0 0 0 0 0 15 0 1 0 658132163 317464576 5749 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899782 0 0 16781312 514 16686808591981576456 0 0 17 5 0 0 0
/proc/7722/statm: 77506 5749 371 364 0 13415 0
[pid=7724] ppid=7722 vsize=3808 CPUtime=5.61 cores=1,3,5,7
/proc/7724/stat : 7724 (march_hi) R 7722 7724 7724 0 -1 4202496 910 0 0 0 561 0 0 0 25 0 1 0 658132163 3899392 648 33554432000 4194304 4955518 140737408834880 18446744073709551615 4227756 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/7724/statm: 952 648 104 186 0 765 0
[pid=7698/tid=7714] ppid=7696 vsize=320268 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7714/stat : 7714 (python) S 7696 7698 7577 0 -1 4202560 25 933 0 0 0 0 0 0 19 0 5 0 658132160 327954432 7255 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 5 0 0 0
[pid=7698/tid=7717] ppid=7696 vsize=320268 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7717/stat : 7717 (python) S 7696 7698 7577 0 -1 4202560 24 933 0 0 0 0 0 0 21 0 5 0 658132161 327954432 7255 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=7720] ppid=7696 vsize=320268 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7720/stat : 7720 (python) S 7696 7698 7577 0 -1 4202560 23 933 0 0 0 0 0 0 23 0 5 0 658132162 327954432 7255 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=7723] ppid=7696 vsize=320268 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7723/stat : 7723 (python) S 7696 7698 7577 0 -1 4202560 28 933 0 0 0 0 0 0 25 0 5 0 658132163 327954432 7255 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 23.05
Current children cumulated vsize (KiB) 3540556

[startup+12.7007 s]
/proc/loadavg: 2.04 0.46 0.81 9/305 7868
/proc/meminfo: memFree=20000148/32950904 swapFree=67057400/67111528
[pid=7698] ppid=7696 vsize=320268 CPUtime=0.51 cores=1,3,5,7
/proc/7698/stat : 7698 (python) S 7696 7698 7577 0 -1 4202496 12206 933 1 0 40 11 0 0 25 0 5 0 658132098 327954432 7255 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7698/statm: 80067 7255 1873 364 0 15976 0
[pid=7713] ppid=7698 vsize=278476 CPUtime=0.01 cores=1,3,5,7
/proc/7713/stat : 7713 (python) S 7698 7698 7577 0 -1 4202560 892 0 0 0 0 1 0 0 18 0 1 0 658132159 285159424 5575 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899727 0 0 16781312 514 2811354475199955208 0 0 17 5 0 0 0
/proc/7713/statm: 69619 5575 377 364 0 5537 0
[pid=7715] ppid=7713 vsize=1017800 CPUtime=12.07 cores=1,3,5,7
/proc/7715/stat : 7715 (gnovelty+2) R 7713 7715 7715 0 -1 4202496 444 0 0 0 1206 1 0 0 25 0 1 0 658132160 1042227200 181 33554432000 134512640 135663704 4293956256 18446744073709551615 134544218 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7715/statm: 254450 181 113 282 0 254166 0
[pid=7716] ppid=7698 vsize=289344 CPUtime=0.01 cores=1,3,5,7

################
# More data... #
################

[pid=7698/tid=7720] ppid=7696 vsize=371968 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7720/stat : 7720 (python) S 7696 7698 7577 0 -1 4202560 23 933 0 0 0 0 0 0 23 0 10 0 658132162 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=7723] ppid=7696 vsize=371968 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7723/stat : 7723 (python) S 7696 7698 7577 0 -1 4202560 28 933 0 0 0 0 0 0 25 0 10 0 658132163 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 5 0 0 0
[pid=7698/tid=7962] ppid=7696 vsize=371968 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7962/stat : 7962 (python) S 7696 7698 7577 0 -1 4202560 21 933 0 0 0 0 0 0 20 0 10 0 658142081 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 5 0 0 0
[pid=7698/tid=7966] ppid=7696 vsize=371968 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7966/stat : 7966 (python) S 7696 7698 7577 0 -1 4202560 31 933 0 0 0 0 0 0 18 0 10 0 658142083 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=8118] ppid=7696 vsize=371968 CPUtime=0 cores=1,3,5,7
/proc/7698/task/8118/stat : 8118 (python) S 7696 7698 7577 0 -1 4202560 31 933 0 0 0 0 0 0 18 0 10 0 658171532 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=8122] ppid=7696 vsize=371968 CPUtime=0 cores=1,3,5,7
/proc/7698/task/8122/stat : 8122 (python) S 7696 7698 7577 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 10 0 658171838 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=8189] ppid=7696 vsize=371968 CPUtime=0 cores=1,3,5,7
/proc/7698/task/8189/stat : 8189 (python) S 7696 7698 7577 0 -1 4202560 25 933 0 0 0 0 0 0 20 0 10 0 658210560 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4776
Current children cumulated vsize (KiB) 6092632

[startup+1198.7 s]
/proc/loadavg: 8.21 8.03 6.09 9/345 8213
/proc/meminfo: memFree=19031292/32950904 swapFree=67057400/67111528
[pid=7698] ppid=7696 vsize=371968 CPUtime=0.6 cores=1,3,5,7
/proc/7698/stat : 7698 (python) S 7696 7698 7577 0 -1 4202496 17061 933 1 0 48 12 0 0 18 0 10 0 658132098 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/7698/statm: 92992 7391 1884 364 0 28886 0
[pid=7713] ppid=7698 vsize=288720 CPUtime=0.19 cores=1,3,5,7
/proc/7713/stat : 7713 (python) S 7698 7698 7577 0 -1 4202560 1161 0 0 0 9 10 0 0 19 0 2 0 658132159 295649280 5619 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/7713/statm: 72180 5619 416 364 0 8098 0
[pid=7715] ppid=7713 vsize=1017800 CPUtime=99.19 cores=1,3,5,7
/proc/7715/stat : 7715 (gnovelty+2) T 7713 7715 7715 0 -1 4202496 444 0 0 0 9918 1 0 0 25 0 1 0 658132160 1042227200 181 33554432000 134512640 135663704 4293956256 18446744073709551615 134525678 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/7715/statm: 254450 181 113 282 0 254166 0
[pid=7713/tid=7960] ppid=7698 vsize=288720 CPUtime=0 cores=1,3,5,7
/proc/7713/task/7960/stat : 7960 (python) S 7698 7698 7577 0 -1 4202560 42 0 0 0 0 0 0 0 19 0 2 0 658142080 295649280 5619 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 514 18446604464332171968 0 0 -1 1 0 0 0
[pid=7716] ppid=7698 vsize=289344 CPUtime=1.56 cores=1,3,5,7
/proc/7716/stat : 7716 (python) S 7698 7698 7577 0 -1 4202560 1162 0 0 0 81 75 0 0 15 0 2 0 658132160 296288256 5746 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899782 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/7716/statm: 72336 5746 400 364 0 8251 0
[pid=7718] ppid=7716 vsize=123716 CPUtime=702.01 cores=1,3,5,7
/proc/7718/stat : 7718 (march_hi) R 7716 7718 7718 0 -1 4202496 21148 0 0 0 70196 5 0 0 25 0 1 0 658132161 126685184 20883 33554432000 4194304 4955518 140735597678256 18446744073709551615 4286753 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/7718/statm: 30929 20883 104 186 0 30742 0
[pid=7716/tid=8116] ppid=7698 vsize=289344 CPUtime=0 cores=1,3,5,7
/proc/7716/task/8116/stat : 8116 (python) S 7698 7698 7577 0 -1 4202560 44 0 0 0 0 0 0 0 16 0 2 0 658171531 296288256 5746 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 514 18446604454518199360 0 0 -1 3 0 0 0
[pid=7719] ppid=7698 vsize=299228 CPUtime=0.2 cores=1,3,5,7
/proc/7719/stat : 7719 (python) S 7698 7698 7577 0 -1 4202560 1088 0 0 0 12 8 0 0 19 0 2 0 658132162 306409472 5640 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/7719/statm: 74807 5640 400 364 0 10719 0
[pid=7721] ppid=7719 vsize=1017800 CPUtime=98.07 cores=1,3,5,7
/proc/7721/stat : 7721 (gnovelty+2) T 7719 7721 7721 0 -1 4202496 444 0 0 0 9807 0 0 0 25 0 1 0 658132162 1042227200 182 33554432000 134512640 135663704 4287591280 18446744073709551615 134545538 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/7721/statm: 254450 182 113 282 0 254166 0
[pid=7719/tid=7963] ppid=7698 vsize=299228 CPUtime=0 cores=1,3,5,7
/proc/7719/task/7963/stat : 7963 (python) S 7698 7698 7577 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 658142082 306409472 5640 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 514 18446604454518200256 0 0 -1 5 0 0 0
[pid=7722] ppid=7698 vsize=310024 CPUtime=0.86 cores=1,3,5,7
/proc/7722/stat : 7722 (python) S 7698 7698 7577 0 -1 4202560 1153 0 0 0 47 39 0 0 16 0 2 0 658132163 317464576 5779 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/7722/statm: 77506 5779 400 364 0 13415 0
[pid=7724] ppid=7722 vsize=58212 CPUtime=392.53 cores=1,3,5,7
/proc/7724/stat : 7724 (march_hi) T 7722 7724 7724 0 -1 4202496 12686 0 0 0 39251 2 0 0 25 0 1 0 658132163 59609088 12424 33554432000 4194304 4955518 140737408834880 18446744073709551615 4209692 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/7724/statm: 14553 12424 104 186 0 14366 0
[pid=7722/tid=8120] ppid=7698 vsize=310024 CPUtime=0 cores=1,3,5,7
/proc/7722/task/8120/stat : 8120 (python) S 7698 7698 7577 0 -1 4202560 45 0 0 0 0 0 0 0 16 0 2 0 658171837 317464576 5779 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 514 18446604470374573952 0 0 -1 5 0 0 0
[pid=7961] ppid=7698 vsize=320280 CPUtime=1.5 cores=1,3,5,7
/proc/7961/stat : 7961 (python) S 7698 7698 7577 0 -1 4202560 1150 0 0 0 79 71 0 0 17 0 2 0 658142081 327966720 5783 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/7961/statm: 80070 5783 400 364 0 15976 0
[pid=7964] ppid=7961 vsize=123716 CPUtime=683.47 cores=1,3,5,7
/proc/7964/stat : 7964 (march_hi) T 7961 7964 7964 0 -1 4202496 20765 0 0 0 68343 4 0 0 25 0 1 0 658142082 126685184 20504 33554432000 4194304 4955518 140734635185184 18446744073709551615 4261387 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/7964/statm: 30929 20504 104 186 0 30742 0
[pid=7961/tid=8187] ppid=7698 vsize=320280 CPUtime=0 cores=1,3,5,7
/proc/7961/task/8187/stat : 8187 (python) S 7698 7698 7577 0 -1 4202560 46 0 0 0 0 0 0 0 17 0 2 0 658210559 327966720 5783 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 514 18446604456805717888 0 0 -1 5 0 0 0
[pid=7965] ppid=7698 vsize=330536 CPUtime=2.24 cores=1,3,5,7
/proc/7965/stat : 7965 (python) S 7698 7698 7577 0 -1 4202560 1162 0 0 0 120 104 0 0 15 0 2 0 658142083 338468864 5798 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899782 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/7965/statm: 82634 5798 401 364 0 18537 0
[pid=7967] ppid=7965 vsize=123716 CPUtime=993.12 cores=1,3,5,7
/proc/7967/stat : 7967 (march_hi) R 7965 7967 7967 0 -1 4202496 28485 0 0 0 99307 5 0 0 25 0 1 0 658142083 126685184 28225 33554432000 4194304 4955518 140733593331248 18446744073709551615 4238615 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/7967/statm: 30929 28225 104 186 0 30742 0
[pid=7965/tid=8192] ppid=7698 vsize=330536 CPUtime=0 cores=1,3,5,7
/proc/7965/task/8192/stat : 8192 (python) S 7698 7698 7577 0 -1 4202560 45 0 0 0 0 0 0 0 15 0 2 0 658211022 338468864 5798 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 514 18446604457807386432 0 0 -1 1 0 0 0
[pid=8117] ppid=7698 vsize=340448 CPUtime=1.82 cores=1,3,5,7
/proc/8117/stat : 8117 (python) S 7698 7698 7577 0 -1 4202560 1177 0 0 0 99 83 0 0 15 0 2 0 658171532 348618752 5705 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899782 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/8117/statm: 85112 5705 400 364 0 21012 0
[pid=8119] ppid=8117 vsize=123716 CPUtime=799.91 cores=1,3,5,7
/proc/8119/stat : 8119 (march_hi) R 8117 8119 8119 0 -1 4202496 23836 0 0 0 79987 4 0 0 25 0 1 0 658171532 126685184 23574 33554432000 4194304 4955518 140734489565808 18446744073709551615 4224704 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/8119/statm: 30929 23574 104 186 0 30742 0
[pid=8117/tid=8191] ppid=7698 vsize=340448 CPUtime=0 cores=1,3,5,7
/proc/8117/task/8191/stat : 8191 (python) S 7698 7698 7577 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 658210814 348618752 5705 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 514 18446604468713913216 0 0 -1 5 0 0 0
[pid=8121] ppid=7698 vsize=351244 CPUtime=1.25 cores=1,3,5,7
/proc/8121/stat : 8121 (python) S 7698 7698 7577 0 -1 4202560 1206 0 0 0 66 59 0 0 19 0 2 0 658171838 359673856 5850 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8121/statm: 87811 5850 406 364 0 23708 0
[pid=8123] ppid=8121 vsize=182452 CPUtime=585.93 cores=1,3,5,7
/proc/8123/stat : 8123 (kcnfs-2006) T 8121 8123 8123 0 -1 4202496 616 0 0 0 58593 0 0 0 25 0 1 0 658171838 186830848 350 33554432000 134512640 135163436 4287499744 18446744073709551615 134569528 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/8123/statm: 45613 350 77 159 0 45453 0
[pid=8121/tid=8202] ppid=7698 vsize=351244 CPUtime=0 cores=1,3,5,7
/proc/8121/task/8202/stat : 8202 (python) S 7698 7698 7577 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 658230462 359673856 5850 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 514 18446604468713911424 0 0 -1 1 0 0 0
[pid=8188] ppid=7698 vsize=361500 CPUtime=0.97 cores=1,3,5,7
/proc/8188/stat : 8188 (python) S 7698 7698 7577 0 -1 4202560 1050 0 0 0 53 44 0 0 16 0 1 0 658210560 370176000 5848 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899782 0 0 16781312 514 2811354475199955207 0 0 17 5 0 0 0
/proc/8188/statm: 90375 5848 377 364 0 26269 0
[pid=8190] ppid=8188 vsize=58212 CPUtime=412.18 cores=1,3,5,7
/proc/8190/stat : 8190 (march_hi) R 8188 8190 8190 0 -1 4202496 13306 0 0 0 41215 3 0 0 25 0 1 0 658210560 59609088 13040 33554432000 4194304 4955518 140733428955312 18446744073709551615 4224773 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/8190/statm: 14553 13040 104 186 0 14366 0
[pid=7698/tid=7714] ppid=7696 vsize=371968 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7714/stat : 7714 (python) S 7696 7698 7577 0 -1 4202560 25 933 0 0 0 0 0 0 19 0 10 0 658132160 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 5 0 0 0
[pid=7698/tid=7717] ppid=7696 vsize=371968 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7717/stat : 7717 (python) S 7696 7698 7577 0 -1 4202560 28 933 0 0 0 0 0 0 18 0 10 0 658132161 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=7720] ppid=7696 vsize=371968 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7720/stat : 7720 (python) S 7696 7698 7577 0 -1 4202560 23 933 0 0 0 0 0 0 23 0 10 0 658132162 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=7723] ppid=7696 vsize=371968 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7723/stat : 7723 (python) S 7696 7698 7577 0 -1 4202560 28 933 0 0 0 0 0 0 25 0 10 0 658132163 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 5 0 0 0
[pid=7698/tid=7962] ppid=7696 vsize=371968 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7962/stat : 7962 (python) S 7696 7698 7577 0 -1 4202560 21 933 0 0 0 0 0 0 20 0 10 0 658142081 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 5 0 0 0
[pid=7698/tid=7966] ppid=7696 vsize=371968 CPUtime=0 cores=1,3,5,7
/proc/7698/task/7966/stat : 7966 (python) S 7696 7698 7577 0 -1 4202560 31 933 0 0 0 0 0 0 18 0 10 0 658142083 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=8118] ppid=7696 vsize=371968 CPUtime=0 cores=1,3,5,7
/proc/7698/task/8118/stat : 8118 (python) S 7696 7698 7577 0 -1 4202560 31 933 0 0 0 0 0 0 18 0 10 0 658171532 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=8122] ppid=7696 vsize=371968 CPUtime=0 cores=1,3,5,7
/proc/7698/task/8122/stat : 8122 (python) S 7696 7698 7577 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 10 0 658171838 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=8189] ppid=7696 vsize=371968 CPUtime=0 cores=1,3,5,7
/proc/7698/task/8189/stat : 8189 (python) S 7696 7698 7577 0 -1 4202560 25 933 0 0 0 0 0 0 20 0 10 0 658210560 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4777.6
Current children cumulated vsize (KiB) 6092632

[startup+1199.1 s]
/proc/loadavg: 8.21 8.03 6.09 9/345 8213
/proc/meminfo: memFree=19030424/32950904 swapFree=67057400/67111528
[pid=7698] ppid=7696 vsize=371968 CPUtime=1863.9 cores=1,3,5,7
/proc/7698/stat : 7698 (python) S 7696 7698 7577 0 -1 4202496 17071 41772 1 0 48 12 186129 201 17 0 10 0 658132098 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290709823 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/7698/statm: 92992 7391 1884 364 0 28886 0
[pid=7716] ppid=7698 vsize=289344 CPUtime=1.57 cores=1,3,5,7
/proc/7716/stat : 7716 (python) R 7698 7698 7577 0 -1 4202560 1162 0 0 0 81 76 0 0 15 0 2 0 658132160 296288256 5746 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278922887 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/7716/statm: 72337 5746 400 364 0 8252 0
[pid=7718] ppid=7716 vsize=123716 CPUtime=702.39 cores=1,3,5,7
/proc/7718/stat : 7718 (march_hi) R 7716 7718 7718 0 -1 4202496 21157 0 0 0 70234 5 0 0 25 0 1 0 658132161 126685184 20892 33554432000 4194304 4955518 140735597678256 18446744073709551615 4209787 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/7718/statm: 30929 20892 104 186 0 30742 0
[pid=7716/tid=8116] ppid=7698 vsize=289344 CPUtime=0 cores=1,3,5,7
/proc/7716/task/8116/stat : 8116 (python) S 7698 7698 7577 0 -1 4202560 44 0 0 0 0 0 0 0 16 0 2 0 658171531 296288256 5746 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 514 18446604454518199360 0 0 -1 3 0 0 0
[pid=7965] ppid=7698 vsize=330536 CPUtime=2.26 cores=1,3,5,7
/proc/7965/stat : 7965 (python) R 7698 7698 7577 0 -1 4202560 1166 0 0 0 121 105 0 0 15 0 2 0 658142083 338468864 5798 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290709823 0 0 16781824 2 18446744073709551615 0 0 17 1 0 0 0
/proc/7965/statm: 82634 5798 401 364 0 18537 0
[pid=7967] ppid=7965 vsize=0 CPUtime=993.5 cores=1,3,5,7
/proc/7967/stat : 7967 (march_hi) Z 7965 7967 7967 0 -1 4203548 28493 0 0 0 99344 6 0 0 25 0 1 0 658142083 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 3 0 0 0
/proc/7967/statm: 0 0 0 0 0 0 0
[pid=7965/tid=8192] ppid=7698 vsize=330536 CPUtime=0 cores=1,3,5,7
/proc/7965/task/8192/stat : 8192 (python) S 7698 7698 7577 0 -1 4202560 45 0 0 0 0 0 0 0 15 0 2 0 658211022 338468864 5798 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781824 2 18446604457807386432 0 0 -1 1 0 0 0
[pid=8117] ppid=7698 vsize=340448 CPUtime=1.82 cores=1,3,5,7
/proc/8117/stat : 8117 (python) S 7698 7698 7577 0 -1 4202560 1177 0 0 0 99 83 0 0 15 0 2 0 658171532 348618752 5705 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899782 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/8117/statm: 85112 5705 400 364 0 21012 0
[pid=8119] ppid=8117 vsize=123716 CPUtime=800.29 cores=1,3,5,7
/proc/8119/stat : 8119 (march_hi) R 8117 8119 8119 0 -1 4202496 23845 0 0 0 80025 4 0 0 25 0 1 0 658171532 126685184 23583 33554432000 4194304 4955518 140734489565808 18446744073709551615 4227679 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/8119/statm: 30929 23583 104 186 0 30742 0
[pid=8117/tid=8191] ppid=7698 vsize=340448 CPUtime=0 cores=1,3,5,7
/proc/8117/task/8191/stat : 8191 (python) S 7698 7698 7577 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 658210814 348618752 5705 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 514 18446604468713913216 0 0 -1 5 0 0 0
[pid=8188] ppid=7698 vsize=361500 CPUtime=0.97 cores=1,3,5,7
/proc/8188/stat : 8188 (python) S 7698 7698 7577 0 -1 4202560 1050 0 0 0 53 44 0 0 15 0 1 0 658210560 370176000 5848 33554432000 4194304 5683644 140733720802656 18446744073709551615 237278899782 0 0 16781312 514 2811354475199955207 0 0 17 5 0 0 0
/proc/8188/statm: 90375 5848 377 364 0 26269 0
[pid=8190] ppid=8188 vsize=58212 CPUtime=412.58 cores=1,3,5,7
/proc/8190/stat : 8190 (march_hi) R 8188 8190 8190 0 -1 4202496 13317 0 0 0 41255 3 0 0 25 0 1 0 658210560 59609088 13051 33554432000 4194304 4955518 140733428955312 18446744073709551615 4238968 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/8190/statm: 14553 13051 104 186 0 14366 0
[pid=7698/tid=7714] ppid=7696 vsize=371968 CPUtime=1863.3 cores=1,3,5,7
/proc/7698/task/7714/stat : 7714 (python) S 7696 7698 7577 0 -1 4202560 25 41772 0 0 0 0 186129 201 19 0 10 0 658132160 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 5 0 0 0
[pid=7698/tid=7717] ppid=7696 vsize=371968 CPUtime=1863.3 cores=1,3,5,7
/proc/7698/task/7717/stat : 7717 (python) S 7696 7698 7577 0 -1 4202560 28 41772 0 0 0 0 186129 201 18 0 10 0 658132161 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=7720] ppid=7696 vsize=371968 CPUtime=1863.3 cores=1,3,5,7
/proc/7698/task/7720/stat : 7720 (python) S 7696 7698 7577 0 -1 4202560 23 41772 0 0 0 0 186129 201 23 0 10 0 658132162 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=7723] ppid=7696 vsize=371968 CPUtime=1863.3 cores=1,3,5,7
/proc/7698/task/7723/stat : 7723 (python) S 7696 7698 7577 0 -1 4202560 28 41772 0 0 0 0 186129 201 25 0 10 0 658132163 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 5 0 0 0
[pid=7698/tid=7962] ppid=7696 vsize=371968 CPUtime=1863.3 cores=1,3,5,7
/proc/7698/task/7962/stat : 7962 (python) S 7696 7698 7577 0 -1 4202560 21 41772 0 0 0 0 186129 201 20 0 10 0 658142081 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 5 0 0 0
[pid=7698/tid=7966] ppid=7696 vsize=371968 CPUtime=1863.3 cores=1,3,5,7
/proc/7698/task/7966/stat : 7966 (python) S 7696 7698 7577 0 -1 4202560 31 41772 0 0 0 0 186129 201 18 0 10 0 658142083 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=8118] ppid=7696 vsize=371968 CPUtime=1863.3 cores=1,3,5,7
/proc/7698/task/8118/stat : 8118 (python) S 7696 7698 7577 0 -1 4202560 31 41772 0 0 0 0 186129 201 18 0 10 0 658171532 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=8122] ppid=7696 vsize=371968 CPUtime=1863.3 cores=1,3,5,7
/proc/7698/task/8122/stat : 8122 (python) S 7696 7698 7577 0 -1 4202560 28 41772 0 0 0 0 186129 201 20 0 10 0 658171838 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
[pid=7698/tid=8189] ppid=7696 vsize=371968 CPUtime=1863.3 cores=1,3,5,7
/proc/7698/task/8189/stat : 8189 (python) S 7696 7698 7577 0 -1 4202560 25 41772 0 0 0 0 186129 201 20 0 10 0 658210560 380895232 7391 33554432000 4194304 5683644 140733720802656 18446744073709551615 237290703281 0 0 16781312 2 18446604459894986048 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4779.28
Current children cumulated vsize (KiB) 1999440

Child status: 0
Real time (s): 1199.14
CPU time (s): 4779.44
CPU user time (s): 4773.98
CPU system time (s): 5.45817
CPU usage (%): 398.573
Max. virtual memory (cumulated for all children) (KiB): 6092636

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4773.98
system time used= 5.45817
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 154372
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 10495
involuntary context switches= 25490

runsolver used 4.73928 second user time and 10.3494 second system time

The end

Launcher Data 

Begin job on node121 at 2011-05-15 15:15:23
IDJOB=3378405
IDBENCH=82814
IDSOLVER=1859
FILE ID=node121/3378405-1305465323
RUNJOBID= node121-1305465318-7596
PBS_JOBID= 13325054
Free space on /tmp= 73492 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v550-c2343-S1694067460-072.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378405-1305465323/watcher-3378405-1305465323 -o /tmp/evaluation-result-3378405-1305465323/solver-3378405-1305465323 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378405-1305465323.cnf -seed 1051148951 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f2d2b125797e776fb2e7099729714d99
RANDOM SEED=1051148951

node121.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950904 kB
MemFree:      20086380 kB
Buffers:        317908 kB
Cached:        6632212 kB
SwapCached:       7656 kB
Active:        6750344 kB
Inactive:      5756024 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950904 kB
LowFree:      20086380 kB
SwapTotal:    67111528 kB
SwapFree:     67057400 kB
Dirty:           64196 kB
Writeback:           4 kB
AnonPages:     5554356 kB
Mapped:          22552 kB
Slab:           268860 kB
PageTables:      24836 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586980 kB
Committed_AS: 30832984 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73432 MiB
End job on node121 at 2011-05-15 15:35:23