Trace number 3385704

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)UNSAT 4793.13 844.464

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v500-c2130-S1205196501-095.UNKNOWN.cnf
MD5SUMb1bf2be38e1cea16efc5e7effa1a10c7
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark757.265
Satisfiable
(Un)Satisfiability was proved
Number of variables500
Number of clauses2130
Sum of the clauses size6390
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 32130
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.29/0.46	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.29/0.46	c INFO: solving HOME/instance-3385704-1305572436.cnf
0.29/0.46	c INFO: computing task features
0.29/0.46	c INFO: computed variable-clause graph statistics
0.29/0.47	c INFO: computed clause balance statistics
0.37/0.50	c INFO: computed variable balance statistics
0.37/0.54	c INFO: computed small-clause counts
0.37/0.54	c INFO: computed Horn-clause counts
0.37/0.54	c INFO: computed variable graph statistics
0.37/0.54	c INFO: collected features for HOME/instance-3385704-1305572436.cnf in 0.08 s
0.37/0.57	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.57)
0.47/0.60	c INFO: running march_hi@0 for 388 with 4999 remaining (b = 0.35)
0.47/0.63	c INFO: running march_hi@0 for 388 with 4999 remaining (b = 0.31)
0.47/0.66	c INFO: running march_hi@0 for 97 with 4999 remaining (b = 0.10)
0.56/0.81	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.07)
0.59/0.94	c INFO: running march_hi@0 for 485 with 4999 remaining (b = 0.26)
2.41/1.18	c INFO: running march_hi@0 for 485 with 4999 remaining (b = 0.23)
3.23/1.31	c INFO: running march_hi@0 for 485 with 4999 remaining (b = 0.21)
594.66/100.95	c INFO: running march_hi@0 for 485 with 4899 remaining (b = 0.18)
629.89/107.37	c INFO: running march_hi@0 for 291 with 4893 remaining (b = 0.13)
872.21/148.65	c INFO: running kcnfs-2006@0 for 582 with 4851 remaining (b = 0.21)
2654.40/484.29	c INFO: running march_hi@0 for 97 with 4516 remaining (b = 0.04)
3083.37/558.94	c INFO: running march_hi@0 for 97 with 4441 remaining (b = 0.05)
3228.01/583.19	c INFO: running march_hi@388 for 97 with 4417 remaining (b = 0.07)
3274.22/590.96	c INFO: running march_hi@0 for 97 with 4409 remaining (b = 0.04)
3335.60/602.12	c INFO: running march_hi@100 for 97 with 4398 remaining (b = 0.05)
3810.49/681.38	c INFO: running march_hi@0 for 388 with 4319 remaining (b = 0.14)
4074.96/723.44	c INFO: running march_hi@287 for 194 with 4277 remaining (b = 0.10)
4123.59/731.53	c INFO: running march_hi@876 for 1261 with 4268 remaining (b = 0.46)
4205.60/745.12	c INFO: running march_hi@0 for 291 with 4255 remaining (b = 0.12)
4217.83/746.94	c INFO: running march_hi@98 for 291 with 4253 remaining (b = 0.13)
4563.92/807.56	c INFO: running march_hi@194 for 97 with 4192 remaining (b = 0.04)
4725.71/833.12	c INFO: running march_hi@101 for 291 with 4167 remaining (b = 0.11)
4792.72/844.44	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385704-1305572436/watcher-3385704-1305572436 -o /tmp/evaluation-result-3385704-1305572436/solver-3385704-1305572436 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385704-1305572436.cnf -seed 386631547 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.58 10.28 10.51 5/458 31967
/proc/meminfo: memFree=22398268/32951124 swapFree=46575272/67111528
[pid=31967] ppid=31965 vsize=16244 CPUtime=0.01 cores=0-7
/proc/31967/stat : 31967 (python) R 31965 31967 30681 0 -1 4202496 1050 0 0 0 1 0 0 0 25 0 1 0 401434852 16633856 923 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660662016 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/31967/statm: 4061 923 381 364 0 548 0

[startup+0.115555 s]
/proc/loadavg: 8.58 10.28 10.51 5/458 31967
/proc/meminfo: memFree=22398268/32951124 swapFree=46575272/67111528
[pid=31967] ppid=31965 vsize=107432 CPUtime=0.07 cores=0-7
/proc/31967/stat : 31967 (python) S 31965 31967 30681 0 -1 4202496 3403 0 0 0 6 1 0 0 17 0 1 0 401434852 110010368 3059 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660660224 0 2147483391 16781312 2 18446744071699199166 0 0 17 0 0 0 0
/proc/31967/statm: 26858 3059 871 364 0 2089 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 107432

[startup+0.200542 s]
/proc/loadavg: 8.58 10.28 10.51 5/458 31967
/proc/meminfo: memFree=22398268/32951124 swapFree=46575272/67111528
[pid=31967] ppid=31965 vsize=126804 CPUtime=0.15 cores=0-7
/proc/31967/stat : 31967 (python) R 31965 31967 30681 0 -1 4202496 5526 937 1 0 12 3 0 0 18 0 1 0 401434852 129847296 4013 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660660224 0 2147483391 16781312 2 0 0 0 17 0 0 0 0
/proc/31967/statm: 31701 4013 1008 364 0 2996 0
Current children cumulated CPU time (s) 0.15
Current children cumulated vsize (KiB) 126804

[startup+0.300558 s]
/proc/loadavg: 8.58 10.28 10.51 5/458 31967
/proc/meminfo: memFree=22398268/32951124 swapFree=46575272/67111528
[pid=31967] ppid=31965 vsize=190256 CPUtime=0.21 cores=0-7
/proc/31967/stat : 31967 (python) S 31965 31967 30681 0 -1 4202496 6885 937 1 0 16 5 0 0 17 0 1 0 401434852 194822144 4944 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660660224 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/31967/statm: 47564 4944 1422 364 0 3724 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 190256

[startup+0.700455 s]
/proc/loadavg: 8.58 10.28 10.51 5/458 31967
/proc/meminfo: memFree=22398268/32951124 swapFree=46575272/67111528
[pid=31967] ppid=31965 vsize=385704 CPUtime=0.55 cores=0-7
/proc/31967/stat : 31967 (python) R 31965 31967 30681 0 -1 4202496 12541 937 1 0 48 7 0 0 20 0 5 0 401434852 394960896 7402 33554432000 4194304 5683644 140734159159360 18446744073709551615 47964904612979 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/31967/statm: 96426 7402 1880 364 0 32335 0
Current children cumulated CPU time (s) 0.55
Current children cumulated vsize (KiB) 385704

[startup+1.50131 s]
/proc/loadavg: 8.58 10.28 10.51 13/477 31990
/proc/meminfo: memFree=22349988/32951124 swapFree=46575272/67111528
[pid=31967] ppid=31965 vsize=426728 CPUtime=0.65 cores=0-7
/proc/31967/stat : 31967 (python) S 31965 31967 30681 0 -1 4202496 15225 937 1 0 58 7 0 0 24 0 9 0 401434852 436969472 7427 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672488907 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/31967/statm: 106682 7427 1889 364 0 42579 0
[pid=31973] ppid=31967 vsize=278604 CPUtime=0 cores=0-7
/proc/31973/stat : 31973 (python) S 31967 31967 30681 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 401434909 285290496 5585 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682255 0 0 16781312 514 1222434008399347711 0 0 17 1 0 0 0
/proc/31973/statm: 69651 5585 354 364 0 5569 0
[pid=31975] ppid=31973 vsize=1017800 CPUtime=0.91 cores=0-7
/proc/31975/stat : 31975 (gnovelty+2) R 31973 31975 31975 0 -1 4202496 438 0 0 0 91 0 0 0 21 0 1 0 401434909 1042227200 177 33554432000 134512640 135663704 4287905104 18446744073709551615 134545207 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31975/statm: 254450 177 113 282 0 254166 0
[pid=31976] ppid=31967 vsize=289412 CPUtime=0 cores=0-7
/proc/31976/stat : 31976 (python) S 31967 31967 30681 0 -1 4202560 755 0 0 0 0 0 0 0 15 0 1 0 401434912 296357888 5710 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 11808873633864974593 0 0 17 7 0 0 0
/proc/31976/statm: 72353 5710 348 364 0 8268 0
[pid=31978] ppid=31976 vsize=2388 CPUtime=0.87 cores=0-7
/proc/31978/stat : 31978 (march_hi) R 31976 31978 31978 0 -1 4202496 664 0 0 0 87 0 0 0 25 0 1 0 401434912 2445312 402 33554432000 4194304 4955518 140736501572112 18446744073709551615 4209793 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31978/statm: 597 402 104 186 0 410 0
[pid=31979] ppid=31967 vsize=365204 CPUtime=0 cores=0-7
/proc/31979/stat : 31979 (python) S 31967 31967 30681 0 -1 4202560 711 0 0 0 0 0 0 0 15 0 1 0 401434915 373968896 5887 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 17822951549148561666 0 0 17 4 0 0 0
/proc/31979/statm: 91301 5887 348 364 0 27213 0
[pid=31981] ppid=31979 vsize=2320 CPUtime=0.7 cores=0-7
/proc/31981/stat : 31981 (march_hi) R 31979 31981 31981 0 -1 4202496 647 0 0 0 70 0 0 0 24 0 1 0 401434916 2375680 395 33554432000 4194304 4955518 140737302486192 18446744073709551615 4227841 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31981/statm: 580 395 104 186 0 393 0
[pid=31982] ppid=31967 vsize=375460 CPUtime=0 cores=0-7
/proc/31982/stat : 31982 (python) S 31967 31967 30681 0 -1 4202560 722 0 0 0 0 0 0 0 17 0 1 0 401434918 384471040 5848 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 9895582664046379266 0 0 17 0 0 0 0
/proc/31982/statm: 93865 5848 348 364 0 29774 0
[pid=31984] ppid=31982 vsize=2184 CPUtime=0.51 cores=0-7
/proc/31984/stat : 31984 (march_hi) R 31982 31984 31984 0 -1 4202496 619 0 0 0 51 0 0 0 25 0 1 0 401434920 2236416 364 33554432000 4194304 4955518 140733318345056 18446744073709551615 4346998 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31984/statm: 546 364 104 186 0 359 0
[pid=31985] ppid=31967 vsize=385716 CPUtime=0 cores=0-7
/proc/31985/stat : 31985 (python) S 31967 31967 30681 0 -1 4202560 725 0 0 0 0 0 0 0 21 0 1 0 401434933 394973184 5870 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 6145654551929159679 0 0 17 1 0 0 0
/proc/31985/statm: 96429 5870 348 364 0 32335 0
[pid=31989] ppid=31985 vsize=1017800 CPUtime=0.27 cores=0-7
/proc/31989/stat : 31989 (gnovelty+2) R 31985 31989 31989 0 -1 4202496 433 0 0 0 27 0 0 0 23 0 1 0 401434952 1042227200 177 33554432000 134512640 135663704 4287123184 18446744073709551615 134544196 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31989/statm: 254450 177 113 282 0 254166 0
[pid=31987] ppid=31967 vsize=395972 CPUtime=0 cores=0-7
/proc/31987/stat : 31987 (python) S 31967 31967 30681 0 -1 4202560 739 0 0 0 0 0 0 0 18 0 1 0 401434946 405475328 5874 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 12512895327176458499 0 0 17 3 0 0 0
/proc/31987/statm: 98993 5874 348 364 0 34896 0
[pid=31990] ppid=31987 vsize=2184 CPUtime=0.21 cores=0-7
/proc/31990/stat : 31990 (march_hi) R 31987 31990 31990 0 -1 4202496 599 0 0 0 21 0 0 0 24 0 1 0 401434959 2236416 342 33554432000 4194304 4955518 140733576647792 18446744073709551615 4227677 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31990/statm: 546 342 104 186 0 359 0
[pid=31967/tid=31974] ppid=31965 vsize=426728 CPUtime=0 cores=0-7
/proc/31967/task/31974/stat : 31974 (python) S 31965 31967 30681 0 -1 4202560 25 937 0 0 0 0 0 0 18 0 9 0 401434909 436969472 7427 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 0 0 0 0
[pid=31967/tid=31977] ppid=31965 vsize=426728 CPUtime=0 cores=0-7
/proc/31967/task/31977/stat : 31977 (python) S 31965 31967 30681 0 -1 4202560 22 937 0 0 0 0 0 0 18 0 9 0 401434912 436969472 7427 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 7 0 0 0
[pid=31967/tid=31980] ppid=31965 vsize=426728 CPUtime=0 cores=0-7
/proc/31967/task/31980/stat : 31980 (python) S 31965 31967 30681 0 -1 4202560 74 937 0 0 0 0 0 0 20 0 9 0 401434915 436969472 7427 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 1 0 0 0
[pid=31967/tid=31983] ppid=31965 vsize=426728 CPUtime=0 cores=0-7
/proc/31967/task/31983/stat : 31983 (python) S 31965 31967 30681 0 -1 4202560 32 937 0 0 0 0 0 0 21 0 9 0 401434918 436969472 7427 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 1 0 0 0
[pid=31967/tid=31986] ppid=31965 vsize=426728 CPUtime=0 cores=0-7
/proc/31967/task/31986/stat : 31986 (python) S 31965 31967 30681 0 -1 4202560 34 937 0 0 0 0 0 0 20 0 9 0 401434933 436969472 7427 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
[pid=31967/tid=31988] ppid=31965 vsize=426728 CPUtime=0 cores=0-7
/proc/31967/task/31988/stat : 31988 (python) S 31965 31967 30681 0 -1 4202560 30 937 0 0 0 0 0 0 23 0 9 0 401434947 436969472 7427 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4.12
Current children cumulated vsize (KiB) 4561772

[startup+3.10103 s]
/proc/loadavg: 8.58 10.28 10.51 13/483 31996
/proc/meminfo: memFree=22339268/32951124 swapFree=46575272/67111528
[pid=31967] ppid=31965 vsize=426728 CPUtime=0.65 cores=0-7
/proc/31967/stat : 31967 (python) S 31965 31967 30681 0 -1 4202496 15225 937 1 0 58 7 0 0 24 0 9 0 401434852 436969472 7427 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672488907 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/31967/statm: 106682 7427 1889 364 0 42579 0
[pid=31973] ppid=31967 vsize=278604 CPUtime=0 cores=0-7
/proc/31973/stat : 31973 (python) S 31967 31967 30681 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 401434909 285290496 5585 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682255 0 0 16781312 514 1222434008399315202 0 0 17 1 0 0 0
/proc/31973/statm: 69651 5585 354 364 0 5569 0
[pid=31975] ppid=31973 vsize=1017800 CPUtime=2.51 cores=0-7
/proc/31975/stat : 31975 (gnovelty+2) R 31973 31975 31975 0 -1 4202496 438 0 0 0 251 0 0 0 25 0 1 0 401434909 1042227200 177 33554432000 134512640 135663704 4287905104 18446744073709551615 134525684 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31975/statm: 254450 177 113 282 0 254166 0
[pid=31976] ppid=31967 vsize=289412 CPUtime=0 cores=0-7
/proc/31976/stat : 31976 (python) S 31967 31967 30681 0 -1 4202560 758 0 0 0 0 0 0 0 15 0 1 0 401434912 296357888 5710 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 11808873633864974593 0 0 17 7 0 0 0
/proc/31976/statm: 72353 5710 348 364 0 8268 0
[pid=31978] ppid=31976 vsize=2824 CPUtime=2.45 cores=0-7
/proc/31978/stat : 31978 (march_hi) R 31976 31978 31978 0 -1 4202496 735 0 0 0 245 0 0 0 25 0 1 0 401434912 2891776 473 33554432000 4194304 4955518 140736501572112 18446744073709551615 4227664 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31978/statm: 706 473 104 186 0 519 0
[pid=31979] ppid=31967 vsize=365204 CPUtime=0 cores=0-7
/proc/31979/stat : 31979 (python) S 31967 31967 30681 0 -1 4202560 712 0 0 0 0 0 0 0 15 0 1 0 401434915 373968896 5887 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 17822951549148561666 0 0 17 4 0 0 0
/proc/31979/statm: 91301 5887 348 364 0 27213 0
[pid=31981] ppid=31979 vsize=2824 CPUtime=2.3 cores=0-7
/proc/31981/stat : 31981 (march_hi) R 31979 31981 31981 0 -1 4202496 718 0 0 0 230 0 0 0 25 0 1 0 401434916 2891776 466 33554432000 4194304 4955518 140737302486192 18446744073709551615 4346513 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31981/statm: 706 466 104 186 0 519 0
[pid=31982] ppid=31967 vsize=375460 CPUtime=0 cores=0-7
/proc/31982/stat : 31982 (python) S 31967 31967 30681 0 -1 4202560 725 0 0 0 0 0 0 0 15 0 1 0 401434918 384471040 5848 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 9895582664046379266 0 0 17 0 0 0 0
/proc/31982/statm: 93865 5848 348 364 0 29774 0
[pid=31984] ppid=31982 vsize=2572 CPUtime=1.31 cores=0-7
/proc/31984/stat : 31984 (march_hi) R 31982 31984 31984 0 -1 4202496 680 0 0 0 131 0 0 0 25 0 1 0 401434920 2633728 425 33554432000 4194304 4955518 140733318345056 18446744073709551615 4307015 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31984/statm: 643 425 104 186 0 456 0
[pid=31985] ppid=31967 vsize=385716 CPUtime=0 cores=0-7
/proc/31985/stat : 31985 (python) S 31967 31967 30681 0 -1 4202560 725 0 0 0 0 0 0 0 18 0 1 0 401434933 394973184 5870 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 6145654551929127170 0 0 17 1 0 0 0
/proc/31985/statm: 96429 5870 348 364 0 32335 0
[pid=31989] ppid=31985 vsize=1017800 CPUtime=1.07 cores=0-7
/proc/31989/stat : 31989 (gnovelty+2) R 31985 31989 31989 0 -1 4202496 433 0 0 0 107 0 0 0 25 0 1 0 401434952 1042227200 177 33554432000 134512640 135663704 4287123184 18446744073709551615 134525278 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31989/statm: 254450 177 113 282 0 254166 0
[pid=31987] ppid=31967 vsize=395972 CPUtime=0 cores=0-7
/proc/31987/stat : 31987 (python) S 31967 31967 30681 0 -1 4202560 740 0 0 0 0 0 0 0 15 0 1 0 401434946 405475328 5874 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 12512895327176458499 0 0 17 3 0 0 0
/proc/31987/statm: 98993 5874 348 364 0 34896 0
[pid=31990] ppid=31987 vsize=2388 CPUtime=1.01 cores=0-7
/proc/31990/stat : 31990 (march_hi) R 31987 31990 31990 0 -1 4202496 665 0 0 0 101 0 0 0 25 0 1 0 401434959 2445312 408 33554432000 4194304 4955518 140733576647792 18446744073709551615 4224541 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31990/statm: 597 408 104 186 0 410 0
[pid=31991] ppid=31967 vsize=406228 CPUtime=0 cores=0-7
/proc/31991/stat : 31991 (python) S 31967 31967 30681 0 -1 4202560 748 0 0 0 0 0 0 0 15 0 1 0 401434970 415977472 5878 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 16278784224960413954 0 0 17 1 0 0 0
/proc/31991/statm: 101557 5878 348 364 0 37457 0
[pid=31993] ppid=31991 vsize=2388 CPUtime=0.91 cores=0-7
/proc/31993/stat : 31993 (march_hi) R 31991 31993 31993 0 -1 4202496 663 0 0 0 91 0 0 0 25 0 1 0 401434982 2445312 404 33554432000 4194304 4955518 140736320125584 18446744073709551615 4209664 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31993/statm: 597 404 104 186 0 410 0
[pid=31994] ppid=31967 vsize=416484 CPUtime=0 cores=0-7
/proc/31994/stat : 31994 (python) S 31967 31967 30681 0 -1 4202560 786 0 0 0 0 0 0 0 15 0 1 0 401434983 426479616 5890 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 3463624359510245636 0 0 17 5 0 0 0
/proc/31994/statm: 104121 5890 355 364 0 40018 0
[pid=31996] ppid=31994 vsize=2320 CPUtime=0.82 cores=0-7
/proc/31996/stat : 31996 (march_hi) R 31994 31996 31996 0 -1 4202496 664 0 0 0 82 0 0 0 25 0 1 0 401434983 2375680 400 33554432000 4194304 4955518 140736808164832 18446744073709551615 4224658 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/31996/statm: 580 400 104 186 0 393 0
[pid=31967/tid=31974] ppid=31965 vsize=426728 CPUtime=0 cores=0-7
/proc/31967/task/31974/stat : 31974 (python) S 31965 31967 30681 0 -1 4202560 25 937 0 0 0 0 0 0 18 0 9 0 401434909 436969472 7427 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 0 0 0 0
[pid=31967/tid=31977] ppid=31965 vsize=426728 CPUtime=0 cores=0-7
/proc/31967/task/31977/stat : 31977 (python) S 31965 31967 30681 0 -1 4202560 22 937 0 0 0 0 0 0 18 0 9 0 401434912 436969472 7427 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 7 0 0 0
[pid=31967/tid=31980] ppid=31965 vsize=426728 CPUtime=0 cores=0-7
/proc/31967/task/31980/stat : 31980 (python) S 31965 31967 30681 0 -1 4202560 74 937 0 0 0 0 0 0 20 0 9 0 401434915 436969472 7427 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 1 0 0 0
[pid=31967/tid=31983] ppid=31965 vsize=426728 CPUtime=0 cores=0-7
/proc/31967/task/31983/stat : 31983 (python) S 31965 31967 30681 0 -1 4202560 32 937 0 0 0 0 0 0 21 0 9 0 401434918 436969472 7427 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 1 0 0 0
[pid=31967/tid=31986] ppid=31965 vsize=426728 CPUtime=0 cores=0-7
/proc/31967/task/31986/stat : 31986 (python) S 31965 31967 30681 0 -1 4202560 34 937 0 0 0 0 0 0 20 0 9 0 401434933 436969472 7427 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
[pid=31967/tid=31988] ppid=31965 vsize=426728 CPUtime=0 cores=0-7
/proc/31967/task/31988/stat : 31988 (python) S 31965 31967 30681 0 -1 4202560 30 937 0 0 0 0 0 0 23 0 9 0 401434947 436969472 7427 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
[pid=31967/tid=31992] ppid=31965 vsize=426728 CPUtime=0 cores=0-7
/proc/31967/task/31992/stat : 31992 (python) S 31965 31967 30681 0 -1 4202560 35 937 0 0 0 0 0 0 22 0 9 0 401434970 436969472 7427 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
[pid=31967/tid=31995] ppid=31965 vsize=426728 CPUtime=0 cores=0-7
/proc/31967/task/31995/stat : 31995 (python) S 31965 31967 30681 0 -1 4202560 35 937 0 0 0 0 0 0 25 0 9 0 401434983 436969472 7427 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 13.03
Current children cumulated vsize (KiB) 5390724

[startup+6.30147 s]
/proc/loadavg: 8.85 10.31 10.52 14/483 31996
/proc/meminfo: memFree=22336184/32951124 swapFree=46575272/67111528
[pid=31967] ppid=31965 vsize=426728 CPUtime=0.65 cores=0-7
/proc/31967/stat : 31967 (python) S 31965 31967 30681 0 -1 4202496 15225 937 1 0 58 7 0 0 24 0 9 0 401434852 436969472 7427 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672488907 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/31967/statm: 106682 7427 1889 364 0 42579 0

################
# More data... #
################

/proc/31993/statm: 19054 16234 104 186 0 18867 0
[pid=31991/tid=32043] ppid=31967 vsize=406228 CPUtime=0 cores=0-7
/proc/31991/task/32043/stat : 32043 (python) S 31967 31967 30681 0 -1 4202560 72 0 0 0 0 0 0 0 17 0 2 0 401508002 415977472 5938 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 514 18446604447063434240 0 0 -1 0 0 0 0
[pid=31994] ppid=31967 vsize=416484 CPUtime=3.14 cores=0-7
/proc/31994/stat : 31994 (python) S 31967 31967 30681 0 -1 4202560 1060 0 0 0 162 152 0 0 16 0 2 0 401434983 426479616 5942 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/31994/statm: 104121 5942 406 364 0 40018 0
[pid=31996] ppid=31994 vsize=76216 CPUtime=485.4 cores=0-7
/proc/31996/stat : 31996 (march_hi) T 31994 31996 31996 0 -1 4202496 14784 0 0 0 48535 5 0 0 25 0 1 0 401434983 78045184 14520 33554432000 4194304 4955518 140736808164832 18446744073709551615 4227679 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/31996/statm: 19054 14520 104 186 0 18867 0
[pid=31994/tid=32048] ppid=31967 vsize=416484 CPUtime=0 cores=0-7
/proc/31994/task/32048/stat : 32048 (python) S 31967 31967 30681 0 -1 4202560 76 0 0 0 0 0 0 0 16 0 2 0 401518161 426479616 5942 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 514 18446604440589866880 0 0 -1 1 0 0 0
[pid=32010] ppid=31967 vsize=426740 CPUtime=2.69 cores=0-7
/proc/32010/stat : 32010 (python) S 31967 31967 30681 0 -1 4202560 1047 0 0 0 141 128 0 0 15 0 2 0 401444947 436981760 5962 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/32010/statm: 106685 5962 401 364 0 42579 0
[pid=32012] ppid=32010 vsize=76216 CPUtime=495.96 cores=0-7
/proc/32012/stat : 32012 (march_hi) R 32010 32012 32012 0 -1 4202496 15068 0 0 0 49592 4 0 0 25 0 1 0 401444947 78045184 14806 33554432000 4194304 4955518 140734340776416 18446744073709551615 4224648 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/32012/statm: 19054 14806 104 186 0 18867 0
[pid=32010/tid=32046] ppid=31967 vsize=426740 CPUtime=0 cores=0-7
/proc/32010/task/32046/stat : 32046 (python) S 31967 31967 30681 0 -1 4202560 77 0 0 0 0 0 0 0 17 0 2 0 401515605 436981760 5962 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 514 18446604441732399488 0 0 -1 0 0 0 0
[pid=32016] ppid=31967 vsize=436996 CPUtime=2.42 cores=0-7
/proc/32016/stat : 32016 (python) S 31967 31967 30681 0 -1 4202560 858 0 0 0 119 123 0 0 18 0 1 0 401449717 447483904 5948 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 13647243877071945985 0 0 17 1 0 0 0
/proc/32016/statm: 109249 5948 371 364 0 45140 0
[pid=32018] ppid=32016 vsize=182452 CPUtime=517.23 cores=0-7
/proc/32018/stat : 32018 (kcnfs-2006) R 32016 32018 32018 0 -1 4202496 591 0 0 0 51723 0 0 0 25 0 1 0 401449717 186830848 330 33554432000 134512640 135163436 4290216320 18446744073709551615 134559218 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/32018/statm: 45613 330 77 159 0 45453 0
[pid=32039] ppid=31967 vsize=447252 CPUtime=0.61 cores=0-7
/proc/32039/stat : 32039 (python) S 31967 31967 30681 0 -1 4202560 878 0 0 0 33 28 0 0 15 0 1 0 401502990 457986048 5952 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 4741133725683646723 0 0 17 1 0 0 0
/proc/32039/statm: 111813 5952 371 364 0 47701 0
[pid=32041] ppid=32039 vsize=27260 CPUtime=136.07 cores=0-7
/proc/32041/stat : 32041 (march_hi) R 32039 32041 32041 0 -1 4202496 5143 0 0 0 13607 0 0 0 25 0 1 0 401502990 27914240 4878 33554432000 4194304 4955518 140733729511600 18446744073709551615 4224727 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/32041/statm: 6815 4878 104 186 0 6628 0
[pid=31967/tid=31974] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/31974/stat : 31974 (python) S 31965 31967 30681 0 -1 4202560 25 937 0 0 0 0 0 0 18 0 12 0 401434909 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 0 0 0 0
[pid=31967/tid=31977] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/31977/stat : 31977 (python) S 31965 31967 30681 0 -1 4202560 32 937 0 0 0 0 0 0 15 0 12 0 401434912 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 7 0 0 0
[pid=31967/tid=31980] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/31980/stat : 31980 (python) S 31965 31967 30681 0 -1 4202560 83 937 0 0 0 0 0 0 18 0 12 0 401434915 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 1 0 0 0
[pid=31967/tid=31983] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/31983/stat : 31983 (python) S 31965 31967 30681 0 -1 4202560 45 937 0 0 0 0 0 0 18 0 12 0 401434918 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 1 0 0 0
[pid=31967/tid=31986] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/31986/stat : 31986 (python) S 31965 31967 30681 0 -1 4202560 34 937 0 0 0 0 0 0 20 0 12 0 401434933 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
[pid=31967/tid=31988] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/31988/stat : 31988 (python) S 31965 31967 30681 0 -1 4202560 43 937 0 0 0 0 0 0 18 0 12 0 401434947 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
[pid=31967/tid=31992] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/31992/stat : 31992 (python) S 31965 31967 30681 0 -1 4202560 41 937 0 0 0 0 0 0 18 0 12 0 401434970 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
[pid=31967/tid=31995] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/31995/stat : 31995 (python) S 31965 31967 30681 0 -1 4202560 35 937 0 0 0 0 0 0 25 0 12 0 401434983 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
[pid=31967/tid=32011] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/32011/stat : 32011 (python) S 31965 31967 30681 0 -1 4202560 46 937 0 0 0 0 0 0 18 0 12 0 401444947 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
[pid=31967/tid=32017] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/32017/stat : 32017 (python) S 31965 31967 30681 0 -1 4202560 38 937 0 0 0 0 0 0 19 0 12 0 401449717 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 3 0 0 0
[pid=31967/tid=32040] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/32040/stat : 32040 (python) S 31965 31967 30681 0 -1 4202560 40 937 0 0 0 0 0 0 19 0 12 0 401502990 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4790.53
Current children cumulated vsize (KiB) 7540188

[startup+844.402 s]
/proc/loadavg: 11.19 11.16 10.95 13/500 32048
/proc/meminfo: memFree=21935656/32951124 swapFree=46575272/67111528
[pid=31967] ppid=31965 vsize=457496 CPUtime=1.15 cores=0-7
/proc/31967/stat : 31967 (python) S 31965 31967 30681 0 -1 4202496 19654 937 1 0 106 9 0 0 18 0 12 0 401434852 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672488907 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/31967/statm: 114374 7508 1896 364 0 50262 0
[pid=31973] ppid=31967 vsize=354384 CPUtime=0.35 cores=0-7
/proc/31973/stat : 31973 (python) S 31967 31967 30681 0 -1 4202560 1089 0 0 0 17 18 0 0 19 0 2 0 401434909 362889216 5648 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/31973/statm: 88596 5648 410 364 0 24514 0
[pid=31975] ppid=31973 vsize=1017800 CPUtime=99.23 cores=0-7
/proc/31975/stat : 31975 (gnovelty+2) T 31973 31975 31975 0 -1 4202496 438 0 0 0 9923 0 0 0 25 0 1 0 401434909 1042227200 177 33554432000 134512640 135663704 4287905104 18446744073709551615 135007680 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/31975/statm: 254450 177 113 282 0 254166 0
[pid=31973/tid=32009] ppid=31967 vsize=354384 CPUtime=0 cores=0-7
/proc/31973/task/32009/stat : 32009 (python) S 31967 31967 30681 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 401444944 362889216 5648 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 514 18446604448902346048 0 0 -1 0 0 0 0
[pid=31976] ppid=31967 vsize=354948 CPUtime=3.19 cores=0-7
/proc/31976/stat : 31976 (python) S 31967 31967 30681 0 -1 4202560 1156 0 0 0 168 151 0 0 15 0 2 0 401434912 363466752 5766 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/31976/statm: 88737 5766 401 364 0 24652 0
[pid=31978] ppid=31976 vsize=76216 CPUtime=537.05 cores=0-7
/proc/31978/stat : 31978 (march_hi) R 31976 31978 31978 0 -1 4202496 16060 0 0 0 53701 4 0 0 25 0 1 0 401434912 78045184 15798 33554432000 4194304 4955518 140736501572112 18446744073709551615 4227502 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31978/statm: 19054 15798 104 186 0 18867 0
[pid=31976/tid=32033] ppid=31967 vsize=354948 CPUtime=0 cores=0-7
/proc/31976/task/32033/stat : 32033 (python) S 31967 31967 30681 0 -1 4202560 44 0 0 0 0 0 0 0 15 0 2 0 401490743 363466752 5766 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 514 18446604441458690752 0 0 -1 7 0 0 0
[pid=31979] ppid=31967 vsize=365204 CPUtime=5.17 cores=0-7
/proc/31979/stat : 31979 (python) R 31967 31967 30681 0 -1 4202560 1020 0 0 0 342 175 0 0 18 0 2 0 401434915 373968896 5940 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672492351 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/31979/statm: 91301 5940 400 364 0 27213 0
[pid=31981] ppid=31979 vsize=0 CPUtime=689.91 cores=0-7
/proc/31981/stat : 31981 (march_hi) R 31979 31981 31981 0 -1 4202500 19669 0 0 0 68985 6 0 0 25 0 1 0 401434916 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 4 0 0 0
/proc/31981/statm: 0 0 0 0 0 0 0
[pid=31979/tid=32030] ppid=31967 vsize=365204 CPUtime=0 cores=0-7
/proc/31979/task/32030/stat : 32030 (python) S 31967 31967 30681 0 -1 4202560 107 0 0 0 0 0 0 0 15 0 2 0 401483278 373968896 5940 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 514 18446604447937792768 0 0 -1 2 0 0 0
[pid=31982] ppid=31967 vsize=375460 CPUtime=2.92 cores=0-7
/proc/31982/stat : 31982 (python) S 31967 31967 30681 0 -1 4202560 992 0 0 0 149 143 0 0 15 0 2 0 401434918 384471040 5901 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/31982/statm: 93865 5901 400 364 0 29774 0
[pid=31984] ppid=31982 vsize=76216 CPUtime=505.54 cores=0-7
/proc/31984/stat : 31984 (march_hi) R 31982 31984 31984 0 -1 4202496 15215 0 0 0 50549 5 0 0 25 0 1 0 401434920 78045184 14960 33554432000 4194304 4955518 140733318345056 18446744073709551615 4346504 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31984/statm: 19054 14960 104 186 0 18867 0
[pid=31982/tid=32013] ppid=31967 vsize=375460 CPUtime=0 cores=0-7
/proc/31982/task/32013/stat : 32013 (python) S 31967 31967 30681 0 -1 4202560 70 0 0 0 0 0 0 0 15 0 2 0 401445586 384471040 5901 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 514 18446604448902347840 0 0 -1 1 0 0 0
[pid=31985] ppid=31967 vsize=385716 CPUtime=0.51 cores=0-7
/proc/31985/stat : 31985 (python) S 31967 31967 30681 0 -1 4202560 986 0 0 0 26 25 0 0 19 0 2 0 401434933 394973184 5923 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/31985/statm: 96429 5923 400 364 0 32335 0
[pid=31989] ppid=31985 vsize=1017800 CPUtime=100.6 cores=0-7
/proc/31989/stat : 31989 (gnovelty+2) T 31985 31989 31989 0 -1 4202496 433 0 0 0 10060 0 0 0 25 0 1 0 401434952 1042227200 177 33554432000 134512640 135663704 4287123184 18446744073709551615 134545576 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/31989/statm: 254450 177 113 282 0 254166 0
[pid=31985/tid=32015] ppid=31967 vsize=385716 CPUtime=0 cores=0-7
/proc/31985/task/32015/stat : 32015 (python) S 31967 31967 30681 0 -1 4202560 70 0 0 0 0 0 0 0 19 0 2 0 401449714 394973184 5923 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 514 18446604441458688064 0 0 -1 1 0 0 0
[pid=31987] ppid=31967 vsize=395972 CPUtime=3.19 cores=0-7
/proc/31987/stat : 31987 (python) S 31967 31967 30681 0 -1 4202560 1016 0 0 0 157 162 0 0 15 0 2 0 401434946 405475328 5928 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/31987/statm: 98993 5928 401 364 0 34896 0
[pid=31990] ppid=31987 vsize=90916 CPUtime=643.12 cores=0-7
/proc/31990/stat : 31990 (march_hi) R 31987 31990 31990 0 -1 4202496 18589 0 0 0 64308 4 0 0 25 0 1 0 401434959 93097984 18332 33554432000 4194304 4955518 140733576647792 18446744073709551615 4286687 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31990/statm: 22729 18332 104 186 0 22542 0
[pid=31987/tid=32035] ppid=31967 vsize=395972 CPUtime=0 cores=0-7
/proc/31987/task/32035/stat : 32035 (python) S 31967 31967 30681 0 -1 4202560 72 0 0 0 0 0 0 0 15 0 2 0 401493945 405475328 5928 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 514 18446604448902346944 0 0 -1 2 0 0 0
[pid=31991] ppid=31967 vsize=406228 CPUtime=3.14 cores=0-7
/proc/31991/stat : 31991 (python) S 31967 31967 30681 0 -1 4202560 1055 0 0 0 161 153 0 0 15 0 2 0 401434970 415977472 5938 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/31991/statm: 101557 5938 407 364 0 37457 0
[pid=31993] ppid=31991 vsize=76216 CPUtime=553.33 cores=0-7
/proc/31993/stat : 31993 (march_hi) R 31991 31993 31993 0 -1 4202496 16502 0 0 0 55330 3 0 0 25 0 1 0 401434982 78045184 16243 33554432000 4194304 4955518 140736320125584 18446744073709551615 4209643 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31993/statm: 19054 16243 104 186 0 18867 0
[pid=31991/tid=32043] ppid=31967 vsize=406228 CPUtime=0 cores=0-7
/proc/31991/task/32043/stat : 32043 (python) S 31967 31967 30681 0 -1 4202560 72 0 0 0 0 0 0 0 17 0 2 0 401508002 415977472 5938 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 514 18446604447063434240 0 0 -1 0 0 0 0
[pid=31994] ppid=31967 vsize=416484 CPUtime=3.14 cores=0-7
/proc/31994/stat : 31994 (python) S 31967 31967 30681 0 -1 4202560 1060 0 0 0 162 152 0 0 16 0 2 0 401434983 426479616 5942 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/31994/statm: 104121 5942 406 364 0 40018 0
[pid=31996] ppid=31994 vsize=76216 CPUtime=485.4 cores=0-7
/proc/31996/stat : 31996 (march_hi) T 31994 31996 31996 0 -1 4202496 14784 0 0 0 48535 5 0 0 25 0 1 0 401434983 78045184 14520 33554432000 4194304 4955518 140736808164832 18446744073709551615 4227679 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/31996/statm: 19054 14520 104 186 0 18867 0
[pid=31994/tid=32048] ppid=31967 vsize=416484 CPUtime=0 cores=0-7
/proc/31994/task/32048/stat : 32048 (python) S 31967 31967 30681 0 -1 4202560 76 0 0 0 0 0 0 0 16 0 2 0 401518161 426479616 5942 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 514 18446604440589866880 0 0 -1 1 0 0 0
[pid=32010] ppid=31967 vsize=426740 CPUtime=2.69 cores=0-7
/proc/32010/stat : 32010 (python) S 31967 31967 30681 0 -1 4202560 1047 0 0 0 141 128 0 0 15 0 2 0 401444947 436981760 5962 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/32010/statm: 106685 5962 401 364 0 42579 0
[pid=32012] ppid=32010 vsize=76216 CPUtime=496.16 cores=0-7
/proc/32012/stat : 32012 (march_hi) R 32010 32012 32012 0 -1 4202496 15073 0 0 0 49612 4 0 0 25 0 1 0 401444947 78045184 14811 33554432000 4194304 4955518 140734340776416 18446744073709551615 4206496 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/32012/statm: 19054 14811 104 186 0 18867 0
[pid=32010/tid=32046] ppid=31967 vsize=426740 CPUtime=0 cores=0-7
/proc/32010/task/32046/stat : 32046 (python) S 31967 31967 30681 0 -1 4202560 77 0 0 0 0 0 0 0 17 0 2 0 401515605 436981760 5962 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 514 18446604441732399488 0 0 -1 0 0 0 0
[pid=32016] ppid=31967 vsize=436996 CPUtime=2.42 cores=0-7
/proc/32016/stat : 32016 (python) S 31967 31967 30681 0 -1 4202560 858 0 0 0 119 123 0 0 18 0 1 0 401449717 447483904 5948 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 13647243877071945985 0 0 17 1 0 0 0
/proc/32016/statm: 109249 5948 371 364 0 45140 0
[pid=32018] ppid=32016 vsize=182452 CPUtime=517.43 cores=0-7
/proc/32018/stat : 32018 (kcnfs-2006) R 32016 32018 32018 0 -1 4202496 591 0 0 0 51743 0 0 0 25 0 1 0 401449717 186830848 330 33554432000 134512640 135163436 4290216320 18446744073709551615 134585038 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/32018/statm: 45613 330 77 159 0 45453 0
[pid=32039] ppid=31967 vsize=447252 CPUtime=0.61 cores=0-7
/proc/32039/stat : 32039 (python) S 31967 31967 30681 0 -1 4202560 878 0 0 0 33 28 0 0 15 0 1 0 401502990 457986048 5952 33554432000 4194304 5683644 140734159159360 18446744073709551615 268660682310 0 0 16781312 514 4741133725683646723 0 0 17 1 0 0 0
/proc/32039/statm: 111813 5952 371 364 0 47701 0
[pid=32041] ppid=32039 vsize=27260 CPUtime=136.47 cores=0-7
/proc/32041/stat : 32041 (march_hi) R 32039 32041 32041 0 -1 4202496 5156 0 0 0 13647 0 0 0 25 0 1 0 401502990 27914240 4891 33554432000 4194304 4955518 140733729511600 18446744073709551615 4224648 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/32041/statm: 6815 4891 104 186 0 6628 0
[pid=31967/tid=31974] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/31974/stat : 31974 (python) S 31965 31967 30681 0 -1 4202560 25 937 0 0 0 0 0 0 18 0 12 0 401434909 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 0 0 0 0
[pid=31967/tid=31977] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/31977/stat : 31977 (python) S 31965 31967 30681 0 -1 4202560 32 937 0 0 0 0 0 0 15 0 12 0 401434912 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 7 0 0 0
[pid=31967/tid=31980] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/31980/stat : 31980 (python) S 31965 31967 30681 0 -1 4202560 83 937 0 0 0 0 0 0 18 0 12 0 401434915 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 1 0 0 0
[pid=31967/tid=31983] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/31983/stat : 31983 (python) S 31965 31967 30681 0 -1 4202560 45 937 0 0 0 0 0 0 18 0 12 0 401434918 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 1 0 0 0
[pid=31967/tid=31986] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/31986/stat : 31986 (python) S 31965 31967 30681 0 -1 4202560 34 937 0 0 0 0 0 0 20 0 12 0 401434933 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
[pid=31967/tid=31988] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/31988/stat : 31988 (python) S 31965 31967 30681 0 -1 4202560 43 937 0 0 0 0 0 0 18 0 12 0 401434947 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
[pid=31967/tid=31992] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/31992/stat : 31992 (python) S 31965 31967 30681 0 -1 4202560 41 937 0 0 0 0 0 0 18 0 12 0 401434970 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
[pid=31967/tid=31995] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/31995/stat : 31995 (python) S 31965 31967 30681 0 -1 4202560 35 937 0 0 0 0 0 0 25 0 12 0 401434983 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
[pid=31967/tid=32011] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/32011/stat : 32011 (python) S 31965 31967 30681 0 -1 4202560 46 937 0 0 0 0 0 0 18 0 12 0 401444947 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
[pid=31967/tid=32017] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/32017/stat : 32017 (python) S 31965 31967 30681 0 -1 4202560 38 937 0 0 0 0 0 0 19 0 12 0 401449717 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 3 0 0 0
[pid=31967/tid=32040] ppid=31965 vsize=457496 CPUtime=0 cores=0-7
/proc/31967/task/32040/stat : 32040 (python) S 31965 31967 30681 0 -1 4202560 40 937 0 0 0 0 0 0 19 0 12 0 401502990 468475904 7508 33554432000 4194304 5683644 140734159159360 18446744073709551615 268672485809 0 0 16781312 2 18446604448581357632 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4792.72
Current children cumulated vsize (KiB) 7540188

Child status: 0
Real time (s): 844.464
CPU time (s): 4793.13
CPU user time (s): 4780.02
CPU system time (s): 13.109
CPU usage (%): 567.594
Max. virtual memory (cumulated for all children) (KiB): 7631104

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4780.02
system time used= 13.109
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 158792
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 11914
involuntary context switches= 48956

runsolver used 5.9051 second user time and 13.6469 second system time

The end

Launcher Data 

Begin job on node107 at 2011-05-16 21:00:36
IDJOB=3385704
IDBENCH=82969
IDSOLVER=1870
FILE ID=node107/3385704-1305572436
RUNJOBID= node107-1305564966-30695
PBS_JOBID= 13350143
Free space on /tmp= 73240 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v500-c2130-S1205196501-095.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385704-1305572436/watcher-3385704-1305572436 -o /tmp/evaluation-result-3385704-1305572436/solver-3385704-1305572436 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385704-1305572436.cnf -seed 386631547 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= b1bf2be38e1cea16efc5e7effa1a10c7
RANDOM SEED=386631547

node107.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5359.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      22398548 kB
Buffers:        233520 kB
Cached:        2437892 kB
SwapCached:       9812 kB
Active:        8813632 kB
Inactive:      1421452 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      22398548 kB
SwapTotal:    67111528 kB
SwapFree:     46575272 kB
Dirty:           34020 kB
Writeback:           0 kB
AnonPages:     7562068 kB
Mapped:          31108 kB
Slab:           155444 kB
PageTables:      90420 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 134616348 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73212 MiB
End job on node107 at 2011-05-16 21:14:40