Trace number 3385624

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)UNSAT 2979.89 374.345

General information on the benchmark

Namerandom/medium/
unif-k7-r89-v60-c5340-S384085843-009.UNKNOWN.cnf
MD5SUM420ae9a699afd6f172f364b156d160ff
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark362.918
Satisfiable
(Un)Satisfiability was proved
Number of variables60
Number of clauses5340
Sum of the clauses size37380
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 55340

Solver Data 

0.31/0.46	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.31/0.46	c INFO: solving HOME/instance-3385624-1305571151.cnf
0.31/0.46	c INFO: computing task features
0.40/0.50	c INFO: computed variable-clause graph statistics
0.40/0.50	c INFO: computed clause balance statistics
0.50/0.69	c INFO: computed variable balance statistics
0.60/0.76	c INFO: computed small-clause counts
0.60/0.76	c INFO: computed Horn-clause counts
0.60/0.77	c INFO: computed variable graph statistics
0.60/0.77	c INFO: collected features for HOME/instance-3385624-1305571151.cnf in 0.31 s
0.69/0.80	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.53)
0.69/0.83	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.09)
0.69/0.86	c INFO: running adaptg2wsat2009++@0 for 97 with 4999 remaining (b = 0.04)
0.69/0.89	c INFO: running adaptg2wsat2009++@0 for 97 with 4999 remaining (b = 0.03)
0.79/0.92	c INFO: running adaptg2wsat2009++@0 for 97 with 4999 remaining (b = 0.03)
0.79/0.95	c INFO: running march_hi@0 for 485 with 4999 remaining (b = 0.11)
0.79/0.98	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.02)
0.79/1.01	c INFO: running march_hi@0 for 485 with 4999 remaining (b = 0.09)
793.00/100.14	c INFO: running FH@0 for 97 with 4900 remaining (b = 0.02)
793.00/100.18	c INFO: running adaptg2wsat2009++@0 for 97 with 4900 remaining (b = 0.01)
793.62/100.27	c INFO: running march_hi@0 for 485 with 4900 remaining (b = 0.08)
794.18/100.37	c INFO: running hybridGM3@0 for 194 with 4900 remaining (b = 0.02)
800.46/101.19	c INFO: running adaptg2wsat2009++@0 for 97 with 4899 remaining (b = 0.01)
801.32/101.23	c INFO: running adaptg2wsat2009++@0 for 97 with 4899 remaining (b = 0.01)
1570.14/197.62	c INFO: running adaptg2wsat2009++@0 for 97 with 4802 remaining (b = 0.01)
1592.49/200.49	c INFO: running adaptg2wsat2009++@98 for 97 with 4800 remaining (b = 0.01)
1602.04/201.61	c INFO: running FH@0 for 97 with 4798 remaining (b = 0.01)
1609.20/202.56	c INFO: running march_hi@0 for 2134 with 4797 remaining (b = 0.24)
2346.17/294.96	c INFO: running march_hi@0 for 2134 with 4705 remaining (b = 0.23)
2363.72/297.19	c INFO: running march_hi@0 for 388 with 4703 remaining (b = 0.06)
2377.26/298.85	c INFO: running march_hi@0 for 388 with 4701 remaining (b = 0.05)
2401.99/301.97	c INFO: running hybridGM3@100 for 97 with 4698 remaining (b = 0.01)
2979.39/374.32	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385624-1305571151/watcher-3385624-1305571151 -o /tmp/evaluation-result-3385624-1305571151/solver-3385624-1305571151 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385624-1305571151.cnf -seed 1828143021 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.01 0.21 0.07 1/417 4842
/proc/meminfo: memFree=20137952/32950928 swapFree=65730892/67111528
[pid=4842] ppid=4840 vsize=15384 CPUtime=0 cores=0-7
/proc/4842/stat : 4842 (python) S 4840 4842 4295 0 -1 4202496 794 0 0 0 0 0 0 0 25 0 1 0 747345821 15753216 681 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905392128 0 2147483391 16781312 2 18446744071699207358 0 0 17 2 0 0 0
/proc/4842/statm: 3846 681 322 364 0 333 0

[startup+0.0817399 s]
/proc/loadavg: 1.01 0.21 0.07 1/417 4842
/proc/meminfo: memFree=20137952/32950928 swapFree=65730892/67111528
[pid=4842] ppid=4840 vsize=87708 CPUtime=0.06 cores=0-7
/proc/4842/stat : 4842 (python) S 4840 4842 4295 0 -1 4202496 2638 0 0 0 5 1 0 0 25 0 1 0 747345821 89812992 2354 33554432000 4194304 5683644 140737410913136 18446744073709551615 260900470407 0 2147483391 16781312 2 18446744071699207358 0 0 17 2 0 0 0
/proc/4842/statm: 21927 2354 717 364 0 1623 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 87708

[startup+0.100733 s]
/proc/loadavg: 1.01 0.21 0.07 1/417 4842
/proc/meminfo: memFree=20137952/32950928 swapFree=65730892/67111528
[pid=4842] ppid=4840 vsize=104236 CPUtime=0.06 cores=0-7
/proc/4842/stat : 4842 (python) R 4840 4842 4295 0 -1 4202496 3087 0 0 0 5 1 0 0 25 0 1 0 747345821 106737664 2768 33554432000 4194304 5683644 140737410913136 18446744073709551615 5114063 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/4842/statm: 26059 2768 847 364 0 1807 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 104236

[startup+0.300767 s]
/proc/loadavg: 1.01 0.21 0.07 1/417 4842
/proc/meminfo: memFree=20137952/32950928 swapFree=65730892/67111528
[pid=4842] ppid=4840 vsize=187692 CPUtime=0.23 cores=0-7
/proc/4842/stat : 4842 (python) S 4840 4842 4295 0 -1 4202496 6830 931 1 0 17 6 0 0 18 0 1 0 747345821 192196608 4884 33554432000 4194304 5683644 140737410913136 18446744073709551615 260900470407 0 2147483391 16781312 2 18446744071699207358 0 0 17 2 0 0 0
/proc/4842/statm: 46923 4884 1399 364 0 3724 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 187692

[startup+0.700623 s]
/proc/loadavg: 1.01 0.21 0.07 1/417 4842
/proc/meminfo: memFree=20137952/32950928 swapFree=65730892/67111528
[pid=4842] ppid=4840 vsize=279556 CPUtime=0.6 cores=0-7
/proc/4842/stat : 4842 (python) R 4840 4842 4295 0 -1 4202496 9751 931 1 0 52 8 0 0 18 0 1 0 747345821 286265344 7274 33554432000 4194304 5683644 140737410913136 18446744073709551615 47221242767064 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/4842/statm: 69889 7274 1827 364 0 5813 0
Current children cumulated CPU time (s) 0.6
Current children cumulated vsize (KiB) 279556

[startup+1.50148 s]
/proc/loadavg: 1.01 0.21 0.07 9/438 4867
/proc/meminfo: memFree=20081676/32950928 swapFree=65730892/67111528
[pid=4842] ppid=4840 vsize=426876 CPUtime=0.91 cores=0-7
/proc/4842/stat : 4842 (python) S 4840 4842 4295 0 -1 4202496 15997 931 1 0 81 10 0 0 22 0 9 0 747345821 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917220811 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4842/statm: 106719 7437 1891 364 0 42616 0
[pid=4848] ppid=4842 vsize=278704 CPUtime=0 cores=0-7
/proc/4848/stat : 4848 (python) S 4842 4842 4295 0 -1 4202560 744 0 0 0 0 0 0 0 20 0 1 0 747345901 285392896 5614 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414159 0 0 16781312 514 9674771317252947967 0 0 17 1 0 0 0
/proc/4848/statm: 69676 5614 354 364 0 5594 0
[pid=4850] ppid=4848 vsize=1017932 CPUtime=0.69 cores=0-7
/proc/4850/stat : 4850 (gnovelty+2) R 4848 4850 4850 0 -1 4202496 511 0 0 0 69 0 0 0 24 0 1 0 747345902 1042362368 248 33554432000 134512640 135663704 4288511680 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/4850/statm: 254483 248 113 282 0 254199 0
[pid=4851] ppid=4842 vsize=289356 CPUtime=0 cores=0-7
/proc/4851/stat : 4851 (python) S 4842 4842 4295 0 -1 4202560 738 0 0 0 0 0 0 0 21 0 1 0 747345904 296300544 5696 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 11145099288810356735 0 0 17 2 0 0 0
/proc/4851/statm: 72339 5696 348 364 0 8254 0
[pid=4853] ppid=4851 vsize=1017932 CPUtime=0.66 cores=0-7
/proc/4853/stat : 4853 (gnovelty+2) R 4851 4853 4853 0 -1 4202496 508 0 0 0 66 0 0 0 25 0 1 0 747345905 1042362368 247 33554432000 134512640 135663704 4292351104 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/4853/statm: 254483 247 113 282 0 254199 0
[pid=4854] ppid=4842 vsize=299276 CPUtime=0 cores=0-7
/proc/4854/stat : 4854 (python) S 4842 4842 4295 0 -1 4202560 724 0 0 0 0 0 0 0 23 0 1 0 747345907 306458624 5609 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 6079248722535907327 0 0 17 0 0 0 0
/proc/4854/statm: 74819 5609 344 364 0 10731 0
[pid=4856] ppid=4854 vsize=416652 CPUtime=0.63 cores=0-7
/proc/4856/stat : 4856 (adaptg2wsat2009) R 4854 4856 4856 0 -1 4202496 565 0 0 0 63 0 0 0 25 0 1 0 747345908 426651648 303 33554432000 134512640 135107411 4291346592 18446744073709551615 134523317 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/4856/statm: 104163 303 49 146 0 103039 0
[pid=4857] ppid=4842 vsize=310072 CPUtime=0 cores=0-7
/proc/4857/stat : 4857 (python) S 4842 4842 4295 0 -1 4202560 731 0 0 0 0 0 0 0 19 0 1 0 747345911 317513728 5734 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 3752875082452566015 0 0 17 0 0 0 0
/proc/4857/statm: 77518 5734 344 364 0 13427 0
[pid=4859] ppid=4857 vsize=416652 CPUtime=0.6 cores=0-7
/proc/4859/stat : 4859 (adaptg2wsat2009) R 4857 4859 4859 0 -1 4202496 567 0 0 0 60 0 0 0 22 0 1 0 747345911 426651648 303 33554432000 134512640 135107411 4287963360 18446744073709551615 134523323 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/4859/statm: 104163 303 49 146 0 103039 0
[pid=4860] ppid=4842 vsize=320328 CPUtime=0 cores=0-7
/proc/4860/stat : 4860 (python) S 4842 4842 4295 0 -1 4202560 736 0 0 0 0 0 0 0 19 0 1 0 747345914 328015872 5738 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 6417373561966559231 0 0 17 1 0 0 0
/proc/4860/statm: 80082 5738 344 364 0 15988 0
[pid=4862] ppid=4860 vsize=416652 CPUtime=0.57 cores=0-7
/proc/4862/stat : 4862 (adaptg2wsat2009) R 4860 4862 4862 0 -1 4202496 569 0 0 0 57 0 0 0 22 0 1 0 747345914 426651648 302 33554432000 134512640 135107411 4289374384 18446744073709551615 134523302 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/4862/statm: 104163 302 49 146 0 103039 0
[pid=4863] ppid=4842 vsize=330584 CPUtime=0 cores=0-7
/proc/4863/stat : 4863 (python) S 4842 4842 4295 0 -1 4202560 775 0 0 0 0 0 0 0 15 0 1 0 747345917 338518016 5746 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 5607385409030947075 0 0 17 1 0 0 0
/proc/4863/statm: 82646 5746 348 364 0 18549 0
[pid=4865] ppid=4863 vsize=2364 CPUtime=0.53 cores=0-7
/proc/4865/stat : 4865 (march_hi) R 4863 4865 4865 0 -1 4202496 1259 0 0 0 53 0 0 0 23 0 1 0 747345917 2420736 407 33554432000 4194304 4955518 140736809116576 18446744073709551615 4224468 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/4865/statm: 591 407 106 186 0 404 0
[pid=4866] ppid=4842 vsize=406376 CPUtime=0 cores=0-7
/proc/4866/stat : 4866 (python) S 4842 4842 4295 0 -1 4202560 738 0 0 0 0 0 0 0 21 0 1 0 747345920 416129024 5923 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 12135173225738928127 0 0 17 1 0 0 0
/proc/4866/statm: 101594 5923 348 364 0 37494 0
[pid=4842/tid=4849] ppid=4840 vsize=426876 CPUtime=0 cores=0-7
/proc/4842/task/4849/stat : 4849 (python) S 4840 4842 4295 0 -1 4202560 25 931 0 0 0 0 0 0 20 0 9 0 747345902 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 0 0 0 0
[pid=4842/tid=4852] ppid=4840 vsize=426876 CPUtime=0 cores=0-7
/proc/4842/task/4852/stat : 4852 (python) S 4840 4842 4295 0 -1 4202560 22 931 0 0 0 0 0 0 18 0 9 0 747345904 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
[pid=4842/tid=4855] ppid=4840 vsize=426876 CPUtime=0 cores=0-7
/proc/4842/task/4855/stat : 4855 (python) S 4840 4842 4295 0 -1 4202560 25 931 0 0 0 0 0 0 24 0 9 0 747345908 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 0 0 0 0
[pid=4842/tid=4858] ppid=4840 vsize=426876 CPUtime=0 cores=0-7
/proc/4842/task/4858/stat : 4858 (python) S 4840 4842 4295 0 -1 4202560 24 931 0 0 0 0 0 0 20 0 9 0 747345911 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 0 0 0 0
[pid=4842/tid=4861] ppid=4840 vsize=426876 CPUtime=0 cores=0-7
/proc/4842/task/4861/stat : 4861 (python) S 4840 4842 4295 0 -1 4202560 25 931 0 0 0 0 0 0 19 0 9 0 747345914 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 0 0 0 0
[pid=4842/tid=4864] ppid=4840 vsize=426876 CPUtime=0 cores=0-7
/proc/4842/task/4864/stat : 4864 (python) S 4840 4842 4295 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 9 0 747345917 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 7 0 0 0
[pid=4842/tid=4867] ppid=4840 vsize=426876 CPUtime=0 cores=0-7
/proc/4842/task/4867/stat : 4867 (python) S 4840 4842 4295 0 -1 4202560 90 931 0 0 0 0 0 0 21 0 9 0 747345920 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
[pid=4842/tid=4869] ppid=4840 vsize=426876 CPUtime=0 cores=0-7
/proc/4842/task/4869/stat : 4869 (python) S 4840 4842 4295 0 -1 4202560 24 931 0 0 0 0 0 0 22 0 9 0 747345923 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4.59
Current children cumulated vsize (KiB) 5949756

[startup+3.10118 s]
/proc/loadavg: 1.01 0.21 0.07 9/442 4871
/proc/meminfo: memFree=20076312/32950928 swapFree=65730892/67111528
[pid=4842] ppid=4840 vsize=426876 CPUtime=0.91 cores=0-7
/proc/4842/stat : 4842 (python) S 4840 4842 4295 0 -1 4202496 15997 931 1 0 81 10 0 0 22 0 9 0 747345821 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917220811 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4842/statm: 106719 7437 1891 364 0 42616 0
[pid=4848] ppid=4842 vsize=278704 CPUtime=0 cores=0-7
/proc/4848/stat : 4848 (python) S 4842 4842 4295 0 -1 4202560 744 0 0 0 0 0 0 0 18 0 1 0 747345901 285392896 5614 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414159 0 0 16781312 514 9674771317252915458 0 0 17 1 0 0 0
/proc/4848/statm: 69676 5614 354 364 0 5594 0
[pid=4850] ppid=4848 vsize=1017932 CPUtime=2.29 cores=0-7
/proc/4850/stat : 4850 (gnovelty+2) R 4848 4850 4850 0 -1 4202496 511 0 0 0 229 0 0 0 25 0 1 0 747345902 1042362368 248 33554432000 134512640 135663704 4288511680 18446744073709551615 134524739 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/4850/statm: 254483 248 113 282 0 254199 0
[pid=4851] ppid=4842 vsize=289356 CPUtime=0 cores=0-7
/proc/4851/stat : 4851 (python) S 4842 4842 4295 0 -1 4202560 738 0 0 0 0 0 0 0 18 0 1 0 747345904 296300544 5696 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 11145099288810324225 0 0 17 2 0 0 0
/proc/4851/statm: 72339 5696 348 364 0 8254 0
[pid=4853] ppid=4851 vsize=1017932 CPUtime=2.26 cores=0-7
/proc/4853/stat : 4853 (gnovelty+2) R 4851 4853 4853 0 -1 4202496 508 0 0 0 226 0 0 0 25 0 1 0 747345905 1042362368 247 33554432000 134512640 135663704 4292351104 18446744073709551615 134525276 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/4853/statm: 254483 247 113 282 0 254199 0
[pid=4854] ppid=4842 vsize=299276 CPUtime=0 cores=0-7
/proc/4854/stat : 4854 (python) S 4842 4842 4295 0 -1 4202560 743 0 0 0 0 0 0 0 18 0 1 0 747345907 306458624 5613 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 6079248722535874817 0 0 17 0 0 0 0
/proc/4854/statm: 74819 5613 348 364 0 10731 0
[pid=4856] ppid=4854 vsize=416788 CPUtime=2.23 cores=0-7
/proc/4856/stat : 4856 (adaptg2wsat2009) R 4854 4856 4856 0 -1 4202496 626 0 0 0 223 0 0 0 25 0 1 0 747345908 426790912 364 33554432000 134512640 135107411 4291346592 18446744073709551615 134532107 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/4856/statm: 104197 364 59 146 0 103073 0
[pid=4857] ppid=4842 vsize=310072 CPUtime=0 cores=0-7
/proc/4857/stat : 4857 (python) S 4842 4842 4295 0 -1 4202560 749 0 0 0 0 0 0 0 18 0 1 0 747345911 317513728 5738 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 3752875082452533507 0 0 17 0 0 0 0
/proc/4857/statm: 77518 5738 348 364 0 13427 0
[pid=4859] ppid=4857 vsize=416788 CPUtime=2.2 cores=0-7
/proc/4859/stat : 4859 (adaptg2wsat2009) R 4857 4859 4859 0 -1 4202496 627 0 0 0 220 0 0 0 25 0 1 0 747345911 426790912 363 33554432000 134512640 135107411 4287963360 18446744073709551615 134531942 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/4859/statm: 104197 363 59 146 0 103073 0
[pid=4860] ppid=4842 vsize=320328 CPUtime=0 cores=0-7
/proc/4860/stat : 4860 (python) S 4842 4842 4295 0 -1 4202560 755 0 0 0 0 0 0 0 18 0 1 0 747345914 328015872 5742 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 6417373561966526723 0 0 17 1 0 0 0
/proc/4860/statm: 80082 5742 348 364 0 15988 0
[pid=4862] ppid=4860 vsize=416788 CPUtime=2.17 cores=0-7
/proc/4862/stat : 4862 (adaptg2wsat2009) R 4860 4862 4862 0 -1 4202496 629 0 0 0 217 0 0 0 25 0 1 0 747345914 426790912 362 33554432000 134512640 135107411 4289374384 18446744073709551615 134531781 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/4862/statm: 104197 362 59 146 0 103073 0
[pid=4863] ppid=4842 vsize=330584 CPUtime=0 cores=0-7
/proc/4863/stat : 4863 (python) S 4842 4842 4295 0 -1 4202560 777 0 0 0 0 0 0 0 15 0 1 0 747345917 338518016 5746 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 12632262707534332163 0 0 17 1 0 0 0
/proc/4863/statm: 82646 5746 348 364 0 18549 0
[pid=4865] ppid=4863 vsize=2364 CPUtime=2.13 cores=0-7
/proc/4865/stat : 4865 (march_hi) R 4863 4865 4865 0 -1 4202496 1259 0 0 0 213 0 0 0 25 0 1 0 747345917 2420736 407 33554432000 4194304 4955518 140736809116576 18446744073709551615 4285265 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/4865/statm: 591 407 106 186 0 404 0
[pid=4866] ppid=4842 vsize=406376 CPUtime=0 cores=0-7
/proc/4866/stat : 4866 (python) S 4842 4842 4295 0 -1 4202560 738 0 0 0 0 0 0 0 18 0 1 0 747345920 416129024 5923 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 12135173225738895617 0 0 17 1 0 0 0
/proc/4866/statm: 101594 5923 348 364 0 37494 0
[pid=4870] ppid=4866 vsize=1017932 CPUtime=2.08 cores=0-7
/proc/4870/stat : 4870 (gnovelty+2) R 4866 4870 4870 0 -1 4202496 504 0 0 0 208 0 0 0 25 0 1 0 747345923 1042362368 248 33554432000 134512640 135663704 4290712160 18446744073709551615 134525229 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/4870/statm: 254483 248 113 282 0 254199 0
[pid=4868] ppid=4842 vsize=416632 CPUtime=0 cores=0-7
/proc/4868/stat : 4868 (python) S 4842 4842 4295 0 -1 4202560 781 0 0 0 0 0 0 0 15 0 1 0 747345923 426631168 5896 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 16312605202630803715 0 0 17 7 0 0 0
/proc/4868/statm: 104158 5896 354 364 0 40055 0
[pid=4871] ppid=4868 vsize=2364 CPUtime=2.04 cores=0-7
/proc/4871/stat : 4871 (march_hi) R 4868 4871 4871 0 -1 4202496 1258 0 0 0 204 0 0 0 25 0 1 0 747345923 2420736 407 33554432000 4194304 4955518 140737427728464 18446744073709551615 4297477 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/4871/statm: 591 407 106 186 0 404 0
[pid=4842/tid=4849] ppid=4840 vsize=426876 CPUtime=0 cores=0-7
/proc/4842/task/4849/stat : 4849 (python) S 4840 4842 4295 0 -1 4202560 25 931 0 0 0 0 0 0 20 0 9 0 747345902 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 0 0 0 0
[pid=4842/tid=4852] ppid=4840 vsize=426876 CPUtime=0 cores=0-7
/proc/4842/task/4852/stat : 4852 (python) S 4840 4842 4295 0 -1 4202560 22 931 0 0 0 0 0 0 18 0 9 0 747345904 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
[pid=4842/tid=4855] ppid=4840 vsize=426876 CPUtime=0 cores=0-7
/proc/4842/task/4855/stat : 4855 (python) S 4840 4842 4295 0 -1 4202560 25 931 0 0 0 0 0 0 24 0 9 0 747345908 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 0 0 0 0
[pid=4842/tid=4858] ppid=4840 vsize=426876 CPUtime=0 cores=0-7
/proc/4842/task/4858/stat : 4858 (python) S 4840 4842 4295 0 -1 4202560 24 931 0 0 0 0 0 0 20 0 9 0 747345911 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 0 0 0 0
[pid=4842/tid=4861] ppid=4840 vsize=426876 CPUtime=0 cores=0-7
/proc/4842/task/4861/stat : 4861 (python) S 4840 4842 4295 0 -1 4202560 25 931 0 0 0 0 0 0 19 0 9 0 747345914 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 0 0 0 0
[pid=4842/tid=4864] ppid=4840 vsize=426876 CPUtime=0 cores=0-7
/proc/4842/task/4864/stat : 4864 (python) S 4840 4842 4295 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 9 0 747345917 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 7 0 0 0
[pid=4842/tid=4867] ppid=4840 vsize=426876 CPUtime=0 cores=0-7
/proc/4842/task/4867/stat : 4867 (python) S 4840 4842 4295 0 -1 4202560 90 931 0 0 0 0 0 0 21 0 9 0 747345920 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
[pid=4842/tid=4869] ppid=4840 vsize=426876 CPUtime=0 cores=0-7
/proc/4842/task/4869/stat : 4869 (python) S 4840 4842 4295 0 -1 4202560 24 931 0 0 0 0 0 0 22 0 9 0 747345923 437121024 7437 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 18.31
Current children cumulated vsize (KiB) 7387092

################
# More data... #
################

[pid=4842/tid=4907] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4907/stat : 4907 (python) S 4840 4842 4295 0 -1 4202560 35 931 0 0 0 0 0 0 20 0 18 0 747375318 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 1 0 0 0
[pid=4842/tid=4911] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4911/stat : 4911 (python) S 4840 4842 4295 0 -1 4202560 33 931 0 0 0 0 0 0 20 0 18 0 747375541 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
[pid=4842/tid=4914] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4914/stat : 4914 (python) S 4840 4842 4295 0 -1 4202560 34 931 0 0 0 0 0 0 20 0 18 0 747375707 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2976.28
Current children cumulated vsize (KiB) 12959032

[startup+374.302 s]
/proc/loadavg: 8.21 5.85 2.71 10/479 4918
/proc/meminfo: memFree=20002672/32950928 swapFree=65730892/67111528
[pid=4842] ppid=4840 vsize=519180 CPUtime=1.48 cores=0-7
/proc/4842/stat : 4842 (python) S 4840 4842 4295 0 -1 4202496 25211 931 1 0 136 12 0 0 18 0 18 0 747345821 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917220811 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4842/statm: 129795 7653 1910 364 0 65665 0
[pid=4848] ppid=4842 vsize=288948 CPUtime=0.3 cores=0-7
/proc/4848/stat : 4848 (python) S 4842 4842 4295 0 -1 4202560 1097 0 0 0 15 15 0 0 19 0 2 0 747345901 295882752 5675 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917220811 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/4848/statm: 72237 5675 410 364 0 8155 0
[pid=4850] ppid=4848 vsize=1017932 CPUtime=99.31 cores=0-7
/proc/4850/stat : 4850 (gnovelty+2) T 4848 4850 4850 0 -1 4202496 511 0 0 0 9931 0 0 0 25 0 1 0 747345902 1042362368 248 33554432000 134512640 135663704 4288511680 18446744073709551615 134525258 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/4850/statm: 254483 248 113 282 0 254199 0
[pid=4848/tid=4875] ppid=4842 vsize=288948 CPUtime=0 cores=0-7
/proc/4848/task/4875/stat : 4875 (python) S 4842 4842 4295 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 747355833 295882752 5675 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 514 18446604442280374592 0 0 -1 1 0 0 0
[pid=4851] ppid=4842 vsize=289356 CPUtime=0.3 cores=0-7
/proc/4851/stat : 4851 (python) S 4842 4842 4295 0 -1 4202560 1079 0 0 0 17 13 0 0 19 0 2 0 747345904 296300544 5749 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917220811 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/4851/statm: 72339 5749 400 364 0 8254 0
[pid=4853] ppid=4851 vsize=1017932 CPUtime=99.3 cores=0-7
/proc/4853/stat : 4853 (gnovelty+2) T 4851 4853 4853 0 -1 4202496 508 0 0 0 9930 0 0 0 25 0 1 0 747345905 1042362368 247 33554432000 134512640 135663704 4292351104 18446744073709551615 134524548 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/4853/statm: 254483 247 113 282 0 254199 0
[pid=4851/tid=4876] ppid=4842 vsize=289356 CPUtime=0 cores=0-7
/proc/4851/task/4876/stat : 4876 (python) S 4842 4842 4295 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 747355835 296300544 5749 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 514 18446604451212646592 0 0 -1 1 0 0 0
[pid=4854] ppid=4842 vsize=299276 CPUtime=0.62 cores=0-7
/proc/4854/stat : 4854 (python) S 4842 4842 4295 0 -1 4202560 1143 0 0 0 32 30 0 0 18 0 2 0 747345907 306458624 5666 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917220811 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/4854/statm: 74819 5666 400 364 0 10731 0
[pid=4856] ppid=4854 vsize=416788 CPUtime=199.28 cores=0-7
/proc/4856/stat : 4856 (adaptg2wsat2009) T 4854 4856 4856 0 -1 4202496 626 0 0 0 19928 0 0 0 25 0 1 0 747345908 426790912 364 33554432000 134512640 135107411 4291346592 18446744073709551615 134532277 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/4856/statm: 104197 364 59 146 0 103073 0
[pid=4854/tid=4891] ppid=4842 vsize=299276 CPUtime=0 cores=0-7
/proc/4854/task/4891/stat : 4891 (python) S 4842 4842 4295 0 -1 4202560 48 0 0 0 0 0 0 0 18 0 2 0 747355937 306458624 5666 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 514 18446604450033161344 0 0 -1 0 0 0 0
[pid=4857] ppid=4842 vsize=310072 CPUtime=0.62 cores=0-7
/proc/4857/stat : 4857 (python) S 4842 4842 4295 0 -1 4202560 1150 0 0 0 35 27 0 0 18 0 2 0 747345911 317513728 5791 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917220811 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/4857/statm: 77518 5791 400 364 0 13427 0
[pid=4859] ppid=4857 vsize=416788 CPUtime=197.59 cores=0-7
/proc/4859/stat : 4859 (adaptg2wsat2009) T 4857 4859 4859 0 -1 4202496 627 0 0 0 19759 0 0 0 25 0 1 0 747345911 426790912 363 33554432000 134512640 135107411 4287963360 18446744073709551615 134532277 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/4859/statm: 104197 363 59 146 0 103073 0
[pid=4857/tid=4892] ppid=4842 vsize=310072 CPUtime=0 cores=0-7
/proc/4857/task/4892/stat : 4892 (python) S 4842 4842 4295 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 747355940 317513728 5791 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 514 18446604453786477568 0 0 -1 0 0 0 0
[pid=4860] ppid=4842 vsize=385864 CPUtime=0.62 cores=0-7
/proc/4860/stat : 4860 (python) S 4842 4842 4295 0 -1 4202560 1132 0 0 0 33 29 0 0 18 0 2 0 747345914 395124736 5803 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917220811 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/4860/statm: 96466 5803 400 364 0 32372 0
[pid=4862] ppid=4860 vsize=416788 CPUtime=200.01 cores=0-7
/proc/4862/stat : 4862 (adaptg2wsat2009) T 4860 4862 4862 0 -1 4202496 629 0 0 0 20001 0 0 0 25 0 1 0 747345914 426790912 362 33554432000 134512640 135107411 4289374384 18446744073709551615 134532277 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/4862/statm: 104197 362 59 146 0 103073 0
[pid=4860/tid=4883] ppid=4842 vsize=385864 CPUtime=0 cores=0-7
/proc/4860/task/4883/stat : 4883 (python) S 4842 4842 4295 0 -1 4202560 50 0 0 0 0 0 0 0 18 0 2 0 747355845 395124736 5803 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 514 18446604441521952576 0 0 -1 0 0 0 0
[pid=4863] ppid=4842 vsize=330584 CPUtime=372.76 cores=0-7
/proc/4863/stat : 4863 (python) R 4842 4842 4295 0 -1 4202560 1112 1697 0 0 250 91 36934 1 18 0 1 0 747345917 338518016 5776 33554432000 4194304 5683644 140737410913136 18446744073709551615 5187280 0 0 16781312 514 0 0 0 17 5 0 0 0
/proc/4863/statm: 82646 5776 378 364 0 18549 0
[pid=4866] ppid=4842 vsize=406376 CPUtime=0.3 cores=0-7
/proc/4866/stat : 4866 (python) S 4842 4842 4295 0 -1 4202560 1036 0 0 0 16 14 0 0 19 0 2 0 747345920 416129024 5976 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917220811 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/4866/statm: 101594 5976 400 364 0 37494 0
[pid=4870] ppid=4866 vsize=1017932 CPUtime=98.98 cores=0-7
/proc/4870/stat : 4870 (gnovelty+2) T 4866 4870 4870 0 -1 4202496 504 0 0 0 9898 0 0 0 25 0 1 0 747345923 1042362368 248 33554432000 134512640 135663704 4290712160 18446744073709551615 134525243 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/4870/statm: 254483 248 113 282 0 254199 0
[pid=4866/tid=4887] ppid=4842 vsize=406376 CPUtime=0 cores=0-7
/proc/4866/task/4887/stat : 4887 (python) S 4842 4842 4295 0 -1 4202560 114 0 0 0 0 0 0 0 19 0 2 0 747355855 416129024 5976 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 514 18446604447148198848 0 0 -1 2 0 0 0
[pid=4868] ppid=4842 vsize=416632 CPUtime=1.33 cores=0-7
/proc/4868/stat : 4868 (python) S 4842 4842 4295 0 -1 4202560 881 0 0 0 70 63 0 0 15 0 1 0 747345923 426631168 5920 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 5607387011053748483 0 0 17 7 0 0 0
/proc/4868/statm: 104158 5920 378 364 0 40055 0
[pid=4871] ppid=4868 vsize=3932 CPUtime=364.18 cores=0-7
/proc/4871/stat : 4871 (march_hi) R 4868 4871 4871 0 -1 4202496 1681 0 0 0 36418 0 0 0 25 0 1 0 747345923 4026368 786 33554432000 4194304 4955518 140737427728464 18446744073709551615 4235627 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/4871/statm: 983 786 106 186 0 796 0
[pid=4877] ppid=4842 vsize=426888 CPUtime=0.3 cores=0-7
/proc/4877/stat : 4877 (python) S 4842 4842 4295 0 -1 4202560 1018 0 0 0 15 15 0 0 19 0 2 0 747355836 437133312 5971 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917220811 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/4877/statm: 106722 5971 400 364 0 42616 0
[pid=4879] ppid=4877 vsize=246936 CPUtime=99.16 cores=0-7
/proc/4879/stat : 4879 (FH) T 4877 4879 4879 0 -1 4202496 588 0 0 0 9916 0 0 0 25 0 1 0 747355837 252862464 327 33554432000 134512640 134936988 4291003072 18446744073709551615 134552382 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/4879/statm: 61734 327 41 104 0 61238 0
[pid=4877/tid=4897] ppid=4842 vsize=426888 CPUtime=0 cores=0-7
/proc/4877/task/4897/stat : 4897 (python) S 4842 4842 4295 0 -1 4202560 79 0 0 0 0 0 0 0 19 0 2 0 747365867 437133312 5971 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 514 18446604450033160448 0 0 -1 0 0 0 0
[pid=4880] ppid=4842 vsize=437144 CPUtime=0.64 cores=0-7
/proc/4880/stat : 4880 (python) S 4842 4842 4295 0 -1 4202560 1035 0 0 0 34 30 0 0 18 0 2 0 747355839 447635456 5988 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917220811 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/4880/statm: 109286 5988 400 364 0 45177 0
[pid=4882] ppid=4880 vsize=416788 CPUtime=195.66 cores=0-7
/proc/4882/stat : 4882 (adaptg2wsat2009) T 4880 4882 4882 0 -1 4202496 627 0 0 0 19566 0 0 0 25 0 1 0 747355840 426790912 362 33554432000 134512640 135107411 4293818544 18446744073709551615 134531661 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/4882/statm: 104197 362 59 146 0 103073 0
[pid=4880/tid=4896] ppid=4842 vsize=437144 CPUtime=0 cores=0-7
/proc/4880/task/4896/stat : 4896 (python) S 4842 4842 4295 0 -1 4202560 75 0 0 0 0 0 0 0 18 0 2 0 747365579 447635456 5988 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 514 18446604449707337856 0 0 -1 4 0 0 0
[pid=4884] ppid=4842 vsize=447400 CPUtime=0.95 cores=0-7
/proc/4884/stat : 4884 (python) S 4842 4842 4295 0 -1 4202560 866 0 0 0 52 43 0 0 15 0 1 0 747355849 458137600 5985 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 14793269249044349185 0 0 17 0 0 0 0
/proc/4884/statm: 111850 5985 371 364 0 47738 0
[pid=4886] ppid=4884 vsize=3420 CPUtime=274.01 cores=0-7
/proc/4886/stat : 4886 (march_hi) R 4884 4886 4886 0 -1 4202496 1562 0 0 0 27401 0 0 0 25 0 1 0 747355849 3502080 670 33554432000 4194304 4955518 140734754665776 18446744073709551615 4208656 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/4886/statm: 855 670 106 186 0 668 0
[pid=4888] ppid=4842 vsize=457656 CPUtime=0.85 cores=0-7
/proc/4888/stat : 4888 (python) S 4842 4842 4295 0 -1 4202560 1056 0 0 0 45 40 0 0 18 0 2 0 747355858 468639744 6043 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/4888/statm: 114414 6043 400 364 0 50299 0
[pid=4890] ppid=4888 vsize=268424 CPUtime=267.48 cores=0-7
/proc/4890/stat : 4890 (hybridGM3) R 4888 4890 4890 0 -1 4202496 4879 0 0 0 26748 0 0 0 25 0 1 0 747355859 274866176 4610 33554432000 134512640 135204385 4288241440 18446744073709551615 134537440 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/4890/statm: 67106 4610 64 169 0 66936 0
[pid=4888/tid=4909] ppid=4842 vsize=457656 CPUtime=0 cores=0-7
/proc/4888/task/4909/stat : 4909 (python) S 4842 4842 4295 0 -1 4202560 77 0 0 0 0 0 0 0 19 0 2 0 747375536 468639744 6043 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 514 18446604449015812480 0 0 -1 6 0 0 0
[pid=4898] ppid=4842 vsize=467912 CPUtime=0.31 cores=0-7
/proc/4898/stat : 4898 (python) S 4842 4842 4295 0 -1 4202560 1071 0 0 0 17 14 0 0 19 0 2 0 747365983 479141888 6023 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917220811 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/4898/statm: 116978 6023 400 364 0 52860 0
[pid=4900] ppid=4898 vsize=246936 CPUtime=99.69 cores=0-7
/proc/4900/stat : 4900 (FH) T 4898 4900 4900 0 -1 4202496 595 0 0 0 9967 2 0 0 25 0 1 0 747365984 252862464 327 33554432000 134512640 134936988 4288139424 18446744073709551615 134545269 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/4900/statm: 61734 327 41 104 0 61238 0
[pid=4898/tid=4916] ppid=4842 vsize=467912 CPUtime=0 cores=0-7
/proc/4898/task/4916/stat : 4916 (python) S 4842 4842 4295 0 -1 4202560 87 0 0 0 0 0 0 0 19 0 2 0 747376014 479141888 6023 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 514 18446604446866492096 0 0 -1 0 0 0 0
[pid=4901] ppid=4842 vsize=478168 CPUtime=0.7 cores=0-7
/proc/4901/stat : 4901 (python) S 4842 4842 4295 0 -1 4202560 14367 0 0 0 38 32 0 0 15 0 1 0 747366078 489644032 6050 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 6924354800121250052 0 0 17 2 0 0 0
/proc/4901/statm: 119542 6050 372 364 0 55421 0
[pid=4903] ppid=4901 vsize=3080 CPUtime=171.73 cores=0-7
/proc/4903/stat : 4903 (march_hi) R 4901 4903 4903 0 -1 4202496 1466 0 0 0 17173 0 0 0 25 0 1 0 747366078 3153920 566 33554432000 4194304 4955518 140734245505888 18446744073709551615 4235606 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/4903/statm: 770 566 106 186 0 583 0
[pid=4906] ppid=4842 vsize=488424 CPUtime=0.27 cores=0-7
/proc/4906/stat : 4906 (python) S 4842 4842 4295 0 -1 4202560 949 0 0 0 13 14 0 0 15 0 1 0 747375318 500146176 6067 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 14095396024755388674 0 0 17 2 0 0 0
/proc/4906/statm: 122106 6067 379 364 0 57982 0
[pid=4908] ppid=4906 vsize=2620 CPUtime=78.34 cores=0-7
/proc/4908/stat : 4908 (march_hi) R 4906 4908 4908 0 -1 4202496 1312 0 0 0 7834 0 0 0 25 0 1 0 747375318 2682880 452 33554432000 4194304 4955518 140736655318048 18446744073709551615 4297472 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/4908/statm: 655 452 106 186 0 468 0
[pid=4910] ppid=4842 vsize=498680 CPUtime=0.27 cores=0-7
/proc/4910/stat : 4910 (python) S 4842 4842 4295 0 -1 4202560 946 0 0 0 14 13 0 0 15 0 1 0 747375541 510648320 6104 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 8255968934474252545 0 0 17 4 0 0 0
/proc/4910/statm: 124670 6104 380 364 0 60543 0
[pid=4912] ppid=4910 vsize=2620 CPUtime=76.94 cores=0-7
/proc/4912/stat : 4912 (march_hi) R 4910 4912 4912 0 -1 4202496 1307 0 0 0 7694 0 0 0 25 0 1 0 747375541 2682880 451 33554432000 4194304 4955518 140735775666288 18446744073709551615 4224450 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/4912/statm: 655 451 106 186 0 468 0
[pid=4913] ppid=4842 vsize=508936 CPUtime=0.26 cores=0-7
/proc/4913/stat : 4913 (python) S 4842 4842 4295 0 -1 4202560 936 0 0 0 16 10 0 0 16 0 1 0 747375707 521150464 6138 33554432000 4194304 5683644 140737410913136 18446744073709551615 260905414214 0 0 16781312 514 18446744073709519106 0 0 17 0 0 0 0
/proc/4913/statm: 127234 6138 374 364 0 63104 0
[pid=4915] ppid=4913 vsize=2620 CPUtime=74.85 cores=0-7
/proc/4915/stat : 4915 (march_hi) R 4913 4915 4915 0 -1 4202496 1311 0 0 0 7484 1 0 0 25 0 1 0 747375707 2682880 449 33554432000 4194304 4955518 140735092824880 18446744073709551615 4297337 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/4915/statm: 655 449 106 186 0 468 0
[pid=4842/tid=4849] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4849/stat : 4849 (python) S 4840 4842 4295 0 -1 4202560 25 931 0 0 0 0 0 0 20 0 18 0 747345902 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 0 0 0 0
[pid=4842/tid=4852] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4852/stat : 4852 (python) S 4840 4842 4295 0 -1 4202560 22 931 0 0 0 0 0 0 18 0 18 0 747345904 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
[pid=4842/tid=4855] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4855/stat : 4855 (python) S 4840 4842 4295 0 -1 4202560 31 931 0 0 0 0 0 0 18 0 18 0 747345908 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
[pid=4842/tid=4858] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4858/stat : 4858 (python) S 4840 4842 4295 0 -1 4202560 32 931 0 0 0 0 0 0 18 0 18 0 747345911 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 0 0 0 0
[pid=4842/tid=4861] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4861/stat : 4861 (python) S 4840 4842 4295 0 -1 4202560 33 931 0 0 0 0 0 0 17 0 18 0 747345914 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
[pid=4842/tid=4864] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4864/stat : 4864 (python) S 4840 4842 4295 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 18 0 747345917 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 7 0 0 0
[pid=4842/tid=4867] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4867/stat : 4867 (python) S 4840 4842 4295 0 -1 4202560 90 931 0 0 0 0 0 0 21 0 18 0 747345920 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
[pid=4842/tid=4869] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4869/stat : 4869 (python) S 4840 4842 4295 0 -1 4202560 24 931 0 0 0 0 0 0 22 0 18 0 747345923 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
[pid=4842/tid=4878] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4878/stat : 4878 (python) S 4840 4842 4295 0 -1 4202560 38 931 0 0 0 0 0 0 19 0 18 0 747355836 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 1 0 0 0
[pid=4842/tid=4881] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4881/stat : 4881 (python) S 4840 4842 4295 0 -1 4202560 41 931 0 0 0 0 0 0 18 0 18 0 747355840 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
[pid=4842/tid=4885] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4885/stat : 4885 (python) S 4840 4842 4295 0 -1 4202560 34 931 0 0 0 0 0 0 22 0 18 0 747355849 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
[pid=4842/tid=4889] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4889/stat : 4889 (python) S 4840 4842 4295 0 -1 4202560 40 931 0 0 0 0 0 0 18 0 18 0 747355859 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
[pid=4842/tid=4899] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4899/stat : 4899 (python) S 4840 4842 4295 0 -1 4202560 44 931 0 0 0 0 0 0 20 0 18 0 747365983 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 1 0 0 0
[pid=4842/tid=4902] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4902/stat : 4902 (python) S 4840 4842 4295 0 -1 4202560 35 931 0 0 0 0 0 0 20 0 18 0 747366078 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
[pid=4842/tid=4907] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4907/stat : 4907 (python) S 4840 4842 4295 0 -1 4202560 35 931 0 0 0 0 0 0 20 0 18 0 747375318 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 1 0 0 0
[pid=4842/tid=4911] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4911/stat : 4911 (python) S 4840 4842 4295 0 -1 4202560 33 931 0 0 0 0 0 0 20 0 18 0 747375541 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
[pid=4842/tid=4914] ppid=4840 vsize=519180 CPUtime=0 cores=0-7
/proc/4842/task/4914/stat : 4914 (python) S 4840 4842 4295 0 -1 4202560 34 931 0 0 0 0 0 0 20 0 18 0 747375707 531640320 7653 33554432000 4194304 5683644 140737410913136 18446744073709551615 260917217713 0 0 16781312 2 18446604446866490304 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2979.39
Current children cumulated vsize (KiB) 12959032

Child status: 0
Real time (s): 374.345
CPU time (s): 2979.89
CPU user time (s): 2974.6
CPU system time (s): 5.29619
CPU usage (%): 796.028
Max. virtual memory (cumulated for all children) (KiB): 12963732

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2974.6
system time used= 5.29619
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 82411
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 6863
involuntary context switches= 19446

runsolver used 2.45663 second user time and 5.33219 second system time

The end

Launcher Data 

Begin job on node116 at 2011-05-16 20:39:11
IDJOB=3385624
IDBENCH=82889
IDSOLVER=1870
FILE ID=node116/3385624-1305571151
RUNJOBID= node116-1305564965-4309
PBS_JOBID= 13350134
Free space on /tmp= 73040 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k7-r89-v60-c5340-S384085843-009.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385624-1305571151/watcher-3385624-1305571151 -o /tmp/evaluation-result-3385624-1305571151/solver-3385624-1305571151 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385624-1305571151.cnf -seed 1828143021 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 420ae9a699afd6f172f364b156d160ff
RANDOM SEED=1828143021

node116.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      20137736 kB
Buffers:        179368 kB
Cached:        2351544 kB
SwapCached:      12328 kB
Active:       11367080 kB
Inactive:      1201532 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      20137736 kB
SwapTotal:    67111528 kB
SwapFree:     65730892 kB
Dirty:           34136 kB
Writeback:           0 kB
AnonPages:    10035788 kB
Mapped:          20320 kB
Slab:           119452 kB
PageTables:      52716 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 113391240 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73008 MiB
End job on node116 at 2011-05-16 20:45:26