Trace number 3385824

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)UNSAT 3801.88 477.492

General information on the benchmark

Namerandom/medium/
unif-k7-r89-v60-c5340-S2117313371-002.UNKNOWN.cnf
MD5SUM80ed5c7022fbd1c0bf0af45d87bcb3f8
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark373.874
Satisfiable
(Un)Satisfiability was proved
Number of variables60
Number of clauses5340
Sum of the clauses size37380
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 55340

Solver Data 

0.31/0.46	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.31/0.47	c INFO: solving HOME/instance-3385824-1305575558.cnf
0.31/0.47	c INFO: computing task features
0.40/0.51	c INFO: computed variable-clause graph statistics
0.40/0.51	c INFO: computed clause balance statistics
0.50/0.69	c INFO: computed variable balance statistics
0.60/0.77	c INFO: computed small-clause counts
0.60/0.77	c INFO: computed Horn-clause counts
0.60/0.77	c INFO: computed variable graph statistics
0.60/0.77	c INFO: collected features for HOME/instance-3385824-1305575558.cnf in 0.31 s
0.70/0.80	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.55)
0.70/0.83	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.10)
0.70/0.86	c INFO: running adaptg2wsat2009++@0 for 97 with 4999 remaining (b = 0.05)
0.70/0.89	c INFO: running adaptg2wsat2009++@0 for 97 with 4999 remaining (b = 0.03)
0.80/0.92	c INFO: running adaptg2wsat2009++@0 for 97 with 4999 remaining (b = 0.03)
0.80/0.95	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.02)
0.80/0.99	c INFO: running FH@0 for 97 with 4999 remaining (b = 0.02)
1.65/1.02	c INFO: running adaptg2wsat2009++@0 for 97 with 4999 remaining (b = 0.01)
793.45/100.21	c INFO: running march_hi@0 for 2134 with 4900 remaining (b = 0.30)
794.16/100.33	c INFO: running march_hi@0 for 2134 with 4900 remaining (b = 0.28)
801.45/101.22	c INFO: running march_hi@0 for 388 with 4899 remaining (b = 0.11)
801.45/101.26	c INFO: running march_hi@0 for 388 with 4899 remaining (b = 0.09)
802.00/101.31	c INFO: running march_hi@0 for 485 with 4899 remaining (b = 0.09)
802.00/101.35	c INFO: running march_hi@0 for 485 with 4899 remaining (b = 0.07)
802.48/101.41	c INFO: running adaptg2wsat2009++@0 for 97 with 4899 remaining (b = 0.01)
802.48/101.45	c INFO: running adaptg2wsat2009++@0 for 97 with 4899 remaining (b = 0.01)
1555.62/195.84	c INFO: running adaptg2wsat2009++@0 for 97 with 4804 remaining (b = 0.01)
1595.35/200.82	c INFO: running adaptg2wsat2009++@99 for 97 with 4799 remaining (b = 0.01)
2332.21/293.23	c INFO: running march_hi@0 for 388 with 4707 remaining (b = 0.05)
2372.08/298.21	c INFO: running FH@0 for 97 with 4702 remaining (b = 0.01)
3172.68/398.62	c INFO: running hybridGM3@0 for 194 with 4601 remaining (b = 0.02)
3801.17/477.47	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385824-1305575558/watcher-3385824-1305575558 -o /tmp/evaluation-result-3385824-1305575558/solver-3385824-1305575558 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385824-1305575558.cnf -seed 1676964751 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.42 6.98 7.73 1/469 31874
/proc/meminfo: memFree=10303648/32951124 swapFree=56316216/67111528
[pid=31874] ppid=31872 vsize=16244 CPUtime=0.01 cores=0-7
/proc/31874/stat : 31874 (python) R 31872 31874 31027 0 -1 4202496 1051 0 0 0 1 0 0 0 24 0 1 0 669158211 16633856 923 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436016896 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/31874/statm: 4061 923 381 364 0 548 0

[startup+0.023831 s]
/proc/loadavg: 5.42 6.98 7.73 1/469 31874
/proc/meminfo: memFree=10303648/32951124 swapFree=56316216/67111528
[pid=31874] ppid=31872 vsize=16536 CPUtime=0.01 cores=0-7
/proc/31874/stat : 31874 (python) R 31872 31874 31027 0 -1 4202496 1184 0 0 0 1 0 0 0 24 0 1 0 669158211 16932864 1046 33554432000 4194304 5683644 140733271031792 18446744073709551615 212435677893 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/31874/statm: 4134 1046 417 364 0 621 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16536

[startup+0.10075 s]
/proc/loadavg: 5.42 6.98 7.73 1/469 31874
/proc/meminfo: memFree=10303648/32951124 swapFree=56316216/67111528
[pid=31874] ppid=31872 vsize=104228 CPUtime=0.06 cores=0-7
/proc/31874/stat : 31874 (python) R 31872 31874 31027 0 -1 4202496 3066 0 0 0 5 1 0 0 17 0 1 0 669158211 106729472 2748 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436015104 0 2147483391 16781312 2 0 0 0 17 6 0 0 0
/proc/31874/statm: 26057 2748 847 364 0 1805 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 104228

[startup+0.300754 s]
/proc/loadavg: 5.42 6.98 7.73 1/469 31874
/proc/meminfo: memFree=10303648/32951124 swapFree=56316216/67111528
[pid=31874] ppid=31872 vsize=170760 CPUtime=0.22 cores=0-7
/proc/31874/stat : 31874 (python) R 31872 31874 31027 0 -1 4202496 6739 937 1 0 17 5 0 0 17 0 1 0 669158211 174858240 4809 33554432000 4194304 5683644 140733271031792 18446744073709551615 4988475 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/31874/statm: 42689 4808 1336 364 0 3454 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 170760

[startup+0.700671 s]
/proc/loadavg: 5.42 6.98 7.73 1/469 31874
/proc/meminfo: memFree=10303648/32951124 swapFree=56316216/67111528
[pid=31874] ppid=31872 vsize=279556 CPUtime=0.6 cores=0-7
/proc/31874/stat : 31874 (python) R 31872 31874 31027 0 -1 4202496 9752 937 1 0 53 7 0 0 17 0 1 0 669158211 286265344 7275 33554432000 4194304 5683644 140733271031792 18446744073709551615 4531504 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/31874/statm: 69889 7275 1827 364 0 5813 0
Current children cumulated CPU time (s) 0.6
Current children cumulated vsize (KiB) 279556

[startup+1.50148 s]
/proc/loadavg: 5.42 6.98 7.73 9/491 31900
/proc/meminfo: memFree=10250224/32951124 swapFree=56316216/67111528
[pid=31874] ppid=31872 vsize=426808 CPUtime=0.92 cores=0-7
/proc/31874/stat : 31874 (python) S 31872 31874 31027 0 -1 4202496 15834 937 1 0 83 9 0 0 25 0 9 0 669158211 437051392 7456 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/31874/statm: 106702 7456 1891 364 0 42599 0
[pid=31880] ppid=31874 vsize=278704 CPUtime=0 cores=0-7
/proc/31880/stat : 31880 (python) S 31874 31874 31027 0 -1 4202560 732 0 0 0 0 0 0 0 19 0 1 0 669158291 285392896 5615 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037135 0 0 16781312 514 5028260192161103871 0 0 17 7 0 0 0
/proc/31880/statm: 69676 5615 354 364 0 5594 0
[pid=31882] ppid=31880 vsize=1017932 CPUtime=0.68 cores=0-7
/proc/31882/stat : 31882 (gnovelty+2) R 31880 31882 31882 0 -1 4202496 509 0 0 0 68 0 0 0 22 0 1 0 669158292 1042362368 247 33554432000 134512640 135663704 4290834064 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31882/statm: 254483 247 113 282 0 254199 0
[pid=31883] ppid=31874 vsize=289356 CPUtime=0 cores=0-7
/proc/31883/stat : 31883 (python) S 31874 31874 31027 0 -1 4202560 743 0 0 0 0 0 0 0 20 0 1 0 669158294 296300544 5697 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 8428963620087398399 0 0 17 6 0 0 0
/proc/31883/statm: 72339 5697 348 364 0 8254 0
[pid=31885] ppid=31883 vsize=1017932 CPUtime=0.65 cores=0-7
/proc/31885/stat : 31885 (gnovelty+2) R 31883 31885 31885 0 -1 4202496 510 0 0 0 65 0 0 0 24 0 1 0 669158295 1042362368 247 33554432000 134512640 135663704 4291652800 18446744073709551615 134524827 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31885/statm: 254483 247 113 282 0 254199 0
[pid=31886] ppid=31874 vsize=299304 CPUtime=0 cores=0-7
/proc/31886/stat : 31886 (python) S 31874 31874 31027 0 -1 4202560 719 0 0 0 0 0 0 0 23 0 1 0 669158297 306487296 5607 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 15772040206439415807 0 0 17 1 0 0 0
/proc/31886/statm: 74826 5607 344 364 0 10738 0
[pid=31888] ppid=31886 vsize=416652 CPUtime=0.62 cores=0-7
/proc/31888/stat : 31888 (adaptg2wsat2009) R 31886 31888 31888 0 -1 4202496 566 0 0 0 62 0 0 0 25 0 1 0 669158298 426651648 302 33554432000 134512640 135107411 4294228560 18446744073709551615 134523346 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31888/statm: 104163 302 49 146 0 103039 0
[pid=31889] ppid=31874 vsize=310004 CPUtime=0 cores=0-7
/proc/31889/stat : 31889 (python) S 31874 31874 31027 0 -1 4202560 727 0 0 0 0 0 0 0 18 0 1 0 669158300 317444096 5718 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 15772032509858021375 0 0 17 1 0 0 0
/proc/31889/statm: 77501 5718 344 364 0 13410 0
[pid=31891] ppid=31889 vsize=416652 CPUtime=0.59 cores=0-7
/proc/31891/stat : 31891 (adaptg2wsat2009) R 31889 31891 31891 0 -1 4202496 568 0 0 0 59 0 0 0 25 0 1 0 669158301 426651648 304 33554432000 134512640 135107411 4289966704 18446744073709551615 134523298 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/31891/statm: 104163 304 49 146 0 103039 0
[pid=31892] ppid=31874 vsize=385796 CPUtime=0 cores=0-7
/proc/31892/stat : 31892 (python) S 31874 31874 31027 0 -1 4202560 705 0 0 0 0 0 0 0 19 0 1 0 669158303 395055104 5895 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 15772036358148718591 0 0 17 7 0 0 0
/proc/31892/statm: 96449 5895 344 364 0 32355 0
[pid=31894] ppid=31892 vsize=416652 CPUtime=0.56 cores=0-7
/proc/31894/stat : 31894 (adaptg2wsat2009) R 31892 31894 31894 0 -1 4202496 560 0 0 0 56 0 0 0 22 0 1 0 669158304 426651648 302 33554432000 134512640 135107411 4289866848 18446744073709551615 134523261 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31894/statm: 104163 302 49 146 0 103039 0
[pid=31895] ppid=31874 vsize=396052 CPUtime=0 cores=0-7
/proc/31895/stat : 31895 (python) S 31874 31874 31027 0 -1 4202560 733 0 0 0 0 0 0 0 20 0 1 0 669158306 405557248 5866 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 8595360961055948799 0 0 17 4 0 0 0
/proc/31895/statm: 99013 5866 348 364 0 34916 0
[pid=31897] ppid=31895 vsize=1017932 CPUtime=0.51 cores=0-7
/proc/31897/stat : 31897 (gnovelty+2) R 31895 31897 31897 0 -1 4202496 507 0 0 0 51 0 0 0 23 0 1 0 669158307 1042362368 248 33554432000 134512640 135663704 4291714768 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31897/statm: 254483 248 113 282 0 254199 0
[pid=31898] ppid=31874 vsize=406308 CPUtime=0 cores=0-7
/proc/31898/stat : 31898 (python) S 31874 31874 31027 0 -1 4202560 721 0 0 0 0 0 0 0 22 0 1 0 669158310 416059392 5884 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 8369354421821243391 0 0 17 4 0 0 0
/proc/31898/statm: 101577 5884 344 364 0 37477 0
[pid=31900] ppid=31898 vsize=245448 CPUtime=0.44 cores=0-7
/proc/31900/stat : 31900 (FH) R 31898 31900 31900 0 -1 4202496 513 0 0 0 44 0 0 0 25 0 1 0 669158310 251338752 256 33554432000 134512640 134936988 4287090240 18446744073709551615 134523665 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31900/statm: 61362 256 37 104 0 60866 0
[pid=31874/tid=31881] ppid=31872 vsize=426808 CPUtime=0 cores=0-7
/proc/31874/task/31881/stat : 31881 (python) S 31872 31874 31027 0 -1 4202560 26 937 0 0 0 0 0 0 19 0 9 0 669158292 437051392 7456 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 0 0 0 0
[pid=31874/tid=31884] ppid=31872 vsize=426808 CPUtime=0 cores=0-7
/proc/31874/task/31884/stat : 31884 (python) S 31872 31874 31027 0 -1 4202560 22 937 0 0 0 0 0 0 21 0 9 0 669158295 437051392 7456 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 6 0 0 0
[pid=31874/tid=31887] ppid=31872 vsize=426808 CPUtime=0 cores=0-7
/proc/31874/task/31887/stat : 31887 (python) S 31872 31874 31027 0 -1 4202560 22 937 0 0 0 0 0 0 22 0 9 0 669158298 437051392 7456 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 6 0 0 0
[pid=31874/tid=31890] ppid=31872 vsize=426808 CPUtime=0 cores=0-7
/proc/31874/task/31890/stat : 31890 (python) S 31872 31874 31027 0 -1 4202560 23 937 0 0 0 0 0 0 18 0 9 0 669158300 437051392 7456 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 5 0 0 0
[pid=31874/tid=31893] ppid=31872 vsize=426808 CPUtime=0 cores=0-7
/proc/31874/task/31893/stat : 31893 (python) S 31872 31874 31027 0 -1 4202560 78 937 0 0 0 0 0 0 19 0 9 0 669158304 437051392 7456 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 0 0 0 0
[pid=31874/tid=31896] ppid=31872 vsize=426808 CPUtime=0 cores=0-7
/proc/31874/task/31896/stat : 31896 (python) S 31872 31874 31027 0 -1 4202560 34 937 0 0 0 0 0 0 21 0 9 0 669158307 437051392 7456 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 0 0 0 0
[pid=31874/tid=31899] ppid=31872 vsize=426808 CPUtime=0 cores=0-7
/proc/31874/task/31899/stat : 31899 (python) S 31872 31874 31027 0 -1 4202560 35 937 0 0 0 0 0 0 23 0 9 0 669158310 437051392 7456 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4.97
Current children cumulated vsize (KiB) 7341532

[startup+3.10118 s]
/proc/loadavg: 5.42 6.98 7.73 9/494 31903
/proc/meminfo: memFree=10243132/32951124 swapFree=56316216/67111528
[pid=31874] ppid=31872 vsize=426808 CPUtime=0.92 cores=0-7
/proc/31874/stat : 31874 (python) S 31872 31874 31027 0 -1 4202496 15834 937 1 0 83 9 0 0 25 0 9 0 669158211 437051392 7456 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/31874/statm: 106702 7456 1891 364 0 42599 0
[pid=31880] ppid=31874 vsize=278704 CPUtime=0 cores=0-7
/proc/31880/stat : 31880 (python) S 31874 31874 31027 0 -1 4202560 732 0 0 0 0 0 0 0 18 0 1 0 669158291 285392896 5615 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037135 0 0 16781312 514 5028260192161071361 0 0 17 7 0 0 0
/proc/31880/statm: 69676 5615 354 364 0 5594 0
[pid=31882] ppid=31880 vsize=1017932 CPUtime=2.28 cores=0-7
/proc/31882/stat : 31882 (gnovelty+2) R 31880 31882 31882 0 -1 4202496 509 0 0 0 228 0 0 0 25 0 1 0 669158292 1042362368 247 33554432000 134512640 135663704 4290834064 18446744073709551615 134524670 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31882/statm: 254483 247 113 282 0 254199 0
[pid=31883] ppid=31874 vsize=289356 CPUtime=0 cores=0-7
/proc/31883/stat : 31883 (python) S 31874 31874 31027 0 -1 4202560 743 0 0 0 0 0 0 0 18 0 1 0 669158294 296300544 5697 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 8428963620087365891 0 0 17 6 0 0 0
/proc/31883/statm: 72339 5697 348 364 0 8254 0
[pid=31885] ppid=31883 vsize=1017932 CPUtime=2.25 cores=0-7
/proc/31885/stat : 31885 (gnovelty+2) R 31883 31885 31885 0 -1 4202496 510 0 0 0 225 0 0 0 25 0 1 0 669158295 1042362368 247 33554432000 134512640 135663704 4291652800 18446744073709551615 134545183 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31885/statm: 254483 247 113 282 0 254199 0
[pid=31886] ppid=31874 vsize=299304 CPUtime=0 cores=0-7
/proc/31886/stat : 31886 (python) S 31874 31874 31027 0 -1 4202560 738 0 0 0 0 0 0 0 18 0 1 0 669158297 306487296 5611 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 15772040206439383300 0 0 17 1 0 0 0
/proc/31886/statm: 74826 5611 348 364 0 10738 0
[pid=31888] ppid=31886 vsize=416788 CPUtime=2.22 cores=0-7
/proc/31888/stat : 31888 (adaptg2wsat2009) R 31886 31888 31888 0 -1 4202496 626 0 0 0 222 0 0 0 25 0 1 0 669158298 426790912 362 33554432000 134512640 135107411 4294228560 18446744073709551615 134531959 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31888/statm: 104197 362 59 146 0 103073 0
[pid=31889] ppid=31874 vsize=310004 CPUtime=0 cores=0-7
/proc/31889/stat : 31889 (python) S 31874 31874 31027 0 -1 4202560 745 0 0 0 0 0 0 0 18 0 1 0 669158300 317444096 5722 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 15772032509857988865 0 0 17 1 0 0 0
/proc/31889/statm: 77501 5722 348 364 0 13410 0
[pid=31891] ppid=31889 vsize=416788 CPUtime=2.19 cores=0-7
/proc/31891/stat : 31891 (adaptg2wsat2009) R 31889 31891 31891 0 -1 4202496 629 0 0 0 219 0 0 0 25 0 1 0 669158301 426790912 365 33554432000 134512640 135107411 4289966704 18446744073709551615 134531428 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/31891/statm: 104197 365 59 146 0 103073 0
[pid=31892] ppid=31874 vsize=385796 CPUtime=0 cores=0-7
/proc/31892/stat : 31892 (python) S 31874 31874 31027 0 -1 4202560 722 0 0 0 0 0 0 0 18 0 1 0 669158303 395055104 5899 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 15772036358148686084 0 0 17 7 0 0 0
/proc/31892/statm: 96449 5899 348 364 0 32355 0
[pid=31894] ppid=31892 vsize=416788 CPUtime=2.16 cores=0-7
/proc/31894/stat : 31894 (adaptg2wsat2009) R 31892 31894 31894 0 -1 4202496 621 0 0 0 216 0 0 0 25 0 1 0 669158304 426790912 363 33554432000 134512640 135107411 4289866848 18446744073709551615 134531513 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31894/statm: 104197 363 59 146 0 103073 0
[pid=31895] ppid=31874 vsize=396052 CPUtime=0 cores=0-7
/proc/31895/stat : 31895 (python) S 31874 31874 31027 0 -1 4202560 733 0 0 0 0 0 0 0 18 0 1 0 669158306 405557248 5866 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 8595360961055916290 0 0 17 4 0 0 0
/proc/31895/statm: 99013 5866 348 364 0 34916 0
[pid=31897] ppid=31895 vsize=1017932 CPUtime=2.1 cores=0-7
/proc/31897/stat : 31897 (gnovelty+2) R 31895 31897 31897 0 -1 4202496 507 0 0 0 210 0 0 0 25 0 1 0 669158307 1042362368 248 33554432000 134512640 135663704 4291714768 18446744073709551615 134524888 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31897/statm: 254483 248 113 282 0 254199 0
[pid=31898] ppid=31874 vsize=406308 CPUtime=0 cores=0-7
/proc/31898/stat : 31898 (python) S 31874 31874 31027 0 -1 4202560 738 0 0 0 0 0 0 0 18 0 1 0 669158310 416059392 5888 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 8369354421821210882 0 0 17 4 0 0 0
/proc/31898/statm: 101577 5888 348 364 0 37477 0
[pid=31900] ppid=31898 vsize=246932 CPUtime=2.04 cores=0-7
/proc/31900/stat : 31900 (FH) R 31898 31900 31900 0 -1 4202496 585 0 0 0 204 0 0 0 25 0 1 0 669158310 252858368 326 33554432000 134512640 134936988 4287090240 18446744073709551615 134554090 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31900/statm: 61733 326 41 104 0 61237 0
[pid=31901] ppid=31874 vsize=416564 CPUtime=0 cores=0-7
/proc/31901/stat : 31901 (python) S 31874 31874 31027 0 -1 4202560 783 0 0 0 0 0 0 0 18 0 1 0 669158313 426561536 5915 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 13012509287669268738 0 0 17 0 0 0 0
/proc/31901/statm: 104141 5915 354 364 0 40038 0
[pid=31903] ppid=31901 vsize=416788 CPUtime=2.07 cores=0-7
/proc/31903/stat : 31903 (adaptg2wsat2009) R 31901 31903 31903 0 -1 4202496 634 0 0 0 207 0 0 0 25 0 1 0 669158313 426790912 363 33554432000 134512640 135107411 4294147840 18446744073709551615 134532408 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31903/statm: 104197 363 59 146 0 103073 0
[pid=31874/tid=31881] ppid=31872 vsize=426808 CPUtime=0 cores=0-7
/proc/31874/task/31881/stat : 31881 (python) S 31872 31874 31027 0 -1 4202560 26 937 0 0 0 0 0 0 19 0 9 0 669158292 437051392 7456 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 0 0 0 0
[pid=31874/tid=31884] ppid=31872 vsize=426808 CPUtime=0 cores=0-7
/proc/31874/task/31884/stat : 31884 (python) S 31872 31874 31027 0 -1 4202560 22 937 0 0 0 0 0 0 21 0 9 0 669158295 437051392 7456 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 6 0 0 0
[pid=31874/tid=31887] ppid=31872 vsize=426808 CPUtime=0 cores=0-7
/proc/31874/task/31887/stat : 31887 (python) S 31872 31874 31027 0 -1 4202560 22 937 0 0 0 0 0 0 22 0 9 0 669158298 437051392 7456 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 6 0 0 0
[pid=31874/tid=31890] ppid=31872 vsize=426808 CPUtime=0 cores=0-7
/proc/31874/task/31890/stat : 31890 (python) S 31872 31874 31027 0 -1 4202560 23 937 0 0 0 0 0 0 18 0 9 0 669158300 437051392 7456 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 5 0 0 0
[pid=31874/tid=31893] ppid=31872 vsize=426808 CPUtime=0 cores=0-7
/proc/31874/task/31893/stat : 31893 (python) S 31872 31874 31027 0 -1 4202560 78 937 0 0 0 0 0 0 19 0 9 0 669158304 437051392 7456 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 0 0 0 0
[pid=31874/tid=31896] ppid=31872 vsize=426808 CPUtime=0 cores=0-7
/proc/31874/task/31896/stat : 31896 (python) S 31872 31874 31027 0 -1 4202560 34 937 0 0 0 0 0 0 21 0 9 0 669158307 437051392 7456 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 0 0 0 0
[pid=31874/tid=31899] ppid=31872 vsize=426808 CPUtime=0 cores=0-7
/proc/31874/task/31899/stat : 31899 (python) S 31872 31874 31027 0 -1 4202560 35 937 0 0 0 0 0 0 23 0 9 0 669158310 437051392 7456 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 4 0 0 0
[pid=31874/tid=31902] ppid=31872 vsize=426808 CPUtime=0 cores=0-7
/proc/31874/task/31902/stat : 31902 (python) S 31872 31874 31027 0 -1 4202560 35 937 0 0 0 0 0 0 25 0 9 0 669158313 437051392 7456 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 18.23

################
# More data... #
################

/proc/31874/task/31939/stat : 31939 (python) S 31872 31874 31027 0 -1 4202560 42 937 0 0 0 0 0 0 18 0 18 0 669187534 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 0 0 0 0
[pid=31874/tid=31942] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31942/stat : 31942 (python) S 31872 31874 31027 0 -1 4202560 33 937 0 0 0 0 0 0 20 0 18 0 669188032 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 7 0 0 0
[pid=31874/tid=31949] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31949/stat : 31949 (python) S 31872 31874 31027 0 -1 4202560 34 937 0 0 0 0 0 0 20 0 18 0 669198074 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3800.36
Current children cumulated vsize (KiB) 13032720

[startup+477.419 s]
/proc/loadavg: 8.19 7.92 7.88 9/530 31952
/proc/meminfo: memFree=10166420/32951124 swapFree=56316216/67111528
[pid=31874] ppid=31872 vsize=519112 CPUtime=1.47 cores=0-7
/proc/31874/stat : 31874 (python) S 31872 31874 31027 0 -1 4202496 24846 937 1 0 136 11 0 0 19 0 18 0 669158211 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/31874/statm: 129778 7584 1910 364 0 65648 0
[pid=31880] ppid=31874 vsize=288948 CPUtime=0.35 cores=0-7
/proc/31880/stat : 31880 (python) S 31874 31874 31027 0 -1 4202560 1090 0 0 0 18 17 0 0 19 0 2 0 669158291 295882752 5676 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/31880/statm: 72237 5676 410 364 0 8155 0
[pid=31882] ppid=31880 vsize=1017932 CPUtime=99.36 cores=0-7
/proc/31882/stat : 31882 (gnovelty+2) T 31880 31882 31882 0 -1 4202496 509 0 0 0 9936 0 0 0 25 0 1 0 669158292 1042362368 247 33554432000 134512640 135663704 4290834064 18446744073709551615 134524509 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/31882/statm: 254483 247 113 282 0 254199 0
[pid=31880/tid=31907] ppid=31874 vsize=288948 CPUtime=0 cores=0-7
/proc/31880/task/31907/stat : 31907 (python) S 31874 31874 31027 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 669168228 295882752 5676 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 514 18446604445493491712 0 0 -1 7 0 0 0
[pid=31883] ppid=31874 vsize=289356 CPUtime=0.35 cores=0-7
/proc/31883/stat : 31883 (python) S 31874 31874 31027 0 -1 4202560 1101 0 0 0 20 15 0 0 19 0 2 0 669158294 296300544 5750 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/31883/statm: 72339 5750 400 364 0 8254 0
[pid=31885] ppid=31883 vsize=1017932 CPUtime=100.31 cores=0-7
/proc/31885/stat : 31885 (gnovelty+2) T 31883 31885 31885 0 -1 4202496 510 0 0 0 10031 0 0 0 25 0 1 0 669158295 1042362368 247 33554432000 134512640 135663704 4291652800 18446744073709551615 134524925 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/31885/statm: 254483 247 113 282 0 254199 0
[pid=31883/tid=31915] ppid=31874 vsize=289356 CPUtime=0 cores=0-7
/proc/31883/task/31915/stat : 31915 (python) S 31874 31874 31027 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 669168330 296300544 5750 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 514 18446604450580246976 0 0 -1 6 0 0 0
[pid=31886] ppid=31874 vsize=299304 CPUtime=0.69 cores=0-7
/proc/31886/stat : 31886 (python) S 31874 31874 31027 0 -1 4202560 1126 0 0 0 38 31 0 0 18 0 2 0 669158297 306487296 5664 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/31886/statm: 74826 5664 400 364 0 10738 0
[pid=31888] ppid=31886 vsize=416788 CPUtime=194.27 cores=0-7
/proc/31888/stat : 31888 (adaptg2wsat2009) T 31886 31888 31888 0 -1 4202496 626 0 0 0 19427 0 0 0 25 0 1 0 669158298 426790912 362 33554432000 134512640 135107411 4294228560 18446744073709551615 134531822 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/31888/statm: 104197 362 59 146 0 103073 0
[pid=31886/tid=31916] ppid=31874 vsize=299304 CPUtime=0 cores=0-7
/proc/31886/task/31916/stat : 31916 (python) S 31874 31874 31027 0 -1 4202560 49 0 0 0 0 0 0 0 18 0 2 0 669168332 306487296 5664 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 514 18446604452289940224 0 0 -1 2 0 0 0
[pid=31889] ppid=31874 vsize=310004 CPUtime=0.71 cores=0-7
/proc/31889/stat : 31889 (python) S 31874 31874 31027 0 -1 4202560 1151 0 0 0 38 33 0 0 18 0 2 0 669158300 317444096 5775 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/31889/statm: 77501 5775 400 364 0 13410 0
[pid=31891] ppid=31889 vsize=416788 CPUtime=197.25 cores=0-7
/proc/31891/stat : 31891 (adaptg2wsat2009) T 31889 31891 31891 0 -1 4202496 629 0 0 0 19725 0 0 0 25 0 1 0 669158301 426790912 365 33554432000 134512640 135107411 4289966704 18446744073709551615 134532313 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/31891/statm: 104197 365 59 146 0 103073 0
[pid=31889/tid=31920] ppid=31874 vsize=310004 CPUtime=0 cores=0-7
/proc/31889/task/31920/stat : 31920 (python) S 31874 31874 31027 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 669168335 317444096 5775 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 514 18446604452289938432 0 0 -1 1 0 0 0
[pid=31892] ppid=31874 vsize=385796 CPUtime=0.72 cores=0-7
/proc/31892/stat : 31892 (python) S 31874 31874 31027 0 -1 4202560 1020 0 0 0 38 34 0 0 18 0 2 0 669158303 395055104 5952 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/31892/statm: 96449 5952 400 364 0 32355 0
[pid=31894] ppid=31892 vsize=416788 CPUtime=197.65 cores=0-7
/proc/31894/stat : 31894 (adaptg2wsat2009) T 31892 31894 31894 0 -1 4202496 621 0 0 0 19765 0 0 0 25 0 1 0 669158304 426790912 363 33554432000 134512640 135107411 4289866848 18446744073709551615 134531428 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/31894/statm: 104197 363 59 146 0 103073 0
[pid=31892/tid=31911] ppid=31874 vsize=385796 CPUtime=0 cores=0-7
/proc/31892/task/31911/stat : 31911 (python) S 31874 31874 31027 0 -1 4202560 109 0 0 0 0 0 0 0 18 0 2 0 669168241 395055104 5952 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 514 18446604452289939328 0 0 -1 7 0 0 0
[pid=31895] ppid=31874 vsize=396052 CPUtime=0.35 cores=0-7
/proc/31895/stat : 31895 (python) S 31874 31874 31027 0 -1 4202560 985 0 0 0 19 16 0 0 19 0 2 0 669158306 405557248 5919 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/31895/statm: 99013 5919 400 364 0 34916 0
[pid=31897] ppid=31895 vsize=1017932 CPUtime=98.04 cores=0-7
/proc/31897/stat : 31897 (gnovelty+2) T 31895 31897 31897 0 -1 4202496 507 0 0 0 9804 0 0 0 25 0 1 0 669158307 1042362368 248 33554432000 134512640 135663704 4291714768 18446744073709551615 134525715 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/31897/statm: 254483 248 113 282 0 254199 0
[pid=31895/tid=31925] ppid=31874 vsize=396052 CPUtime=0 cores=0-7
/proc/31895/task/31925/stat : 31925 (python) S 31874 31874 31027 0 -1 4202560 65 0 0 0 0 0 0 0 19 0 2 0 669168342 405557248 5919 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 514 18446604446324022080 0 0 -1 4 0 0 0
[pid=31898] ppid=31874 vsize=406308 CPUtime=0.35 cores=0-7
/proc/31898/stat : 31898 (python) S 31874 31874 31027 0 -1 4202560 998 0 0 0 14 21 0 0 19 0 2 0 669158310 416059392 5941 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/31898/statm: 101577 5941 400 364 0 37477 0
[pid=31900] ppid=31898 vsize=246932 CPUtime=100 cores=0-7
/proc/31900/stat : 31900 (FH) T 31898 31900 31900 0 -1 4202496 585 0 0 0 10000 0 0 0 25 0 1 0 669158310 252858368 326 33554432000 134512640 134936988 4287090240 18446744073709551615 134544944 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/31900/statm: 61733 326 41 104 0 61237 0
[pid=31898/tid=31932] ppid=31874 vsize=406308 CPUtime=0 cores=0-7
/proc/31898/task/31932/stat : 31932 (python) S 31874 31874 31027 0 -1 4202560 66 0 0 0 0 0 0 0 19 0 2 0 669168350 416059392 5941 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 514 18446604441976433536 0 0 -1 4 0 0 0
[pid=31901] ppid=31874 vsize=416564 CPUtime=0.71 cores=0-7
/proc/31901/stat : 31901 (python) S 31874 31874 31027 0 -1 4202560 1068 0 0 0 38 33 0 0 16 0 2 0 669158313 426561536 5968 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/31901/statm: 104141 5968 406 364 0 40038 0
[pid=31903] ppid=31901 vsize=416788 CPUtime=194.62 cores=0-7
/proc/31903/stat : 31903 (adaptg2wsat2009) T 31901 31903 31903 0 -1 4202496 634 0 0 0 19462 0 0 0 25 0 1 0 669158313 426790912 363 33554432000 134512640 135107411 4294147840 18446744073709551615 134532121 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/31903/statm: 104197 363 59 146 0 103073 0
[pid=31901/tid=31931] ppid=31874 vsize=416564 CPUtime=0 cores=0-7
/proc/31901/task/31931/stat : 31931 (python) S 31874 31874 31027 0 -1 4202560 86 0 0 0 0 0 0 0 18 0 2 0 669168347 426561536 5968 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 514 18446604443057502272 0 0 -1 5 0 0 0
[pid=31908] ppid=31874 vsize=426820 CPUtime=1.53 cores=0-7
/proc/31908/stat : 31908 (python) S 31874 31874 31027 0 -1 4202560 859 0 0 0 79 74 0 0 15 0 1 0 669168232 437063680 5936 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 5607406381356253441 0 0 17 0 0 0 0
/proc/31908/statm: 106705 5936 371 364 0 42599 0
[pid=31910] ppid=31908 vsize=4696 CPUtime=375.65 cores=0-7
/proc/31910/stat : 31910 (march_hi) R 31908 31910 31910 0 -1 4202496 1694 0 0 0 37565 0 0 0 25 0 1 0 669168232 4808704 799 33554432000 4194304 4955518 140736714399024 18446744073709551615 4297429 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31910/statm: 1174 799 106 186 0 987 0
[pid=31912] ppid=31874 vsize=437076 CPUtime=1.53 cores=0-7
/proc/31912/stat : 31912 (python) S 31874 31874 31027 0 -1 4202560 881 0 0 0 82 71 0 0 15 0 1 0 669168244 447565824 5979 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 3170718859917230342 0 0 17 4 0 0 0
/proc/31912/statm: 109269 5979 371 364 0 45160 0
[pid=31914] ppid=31912 vsize=3932 CPUtime=375.48 cores=0-7
/proc/31914/stat : 31914 (march_hi) R 31912 31914 31914 0 -1 4202496 1691 0 0 0 37548 0 0 0 25 0 1 0 669168245 4026368 798 33554432000 4194304 4955518 140736049971456 18446744073709551615 4208667 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31914/statm: 983 798 106 186 0 796 0
[pid=31917] ppid=31874 vsize=447332 CPUtime=1.53 cores=0-7
/proc/31917/stat : 31917 (python) S 31874 31874 31027 0 -1 4202560 881 0 0 0 78 75 0 0 15 0 1 0 669168333 458067968 5996 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 18143349237482488072 0 0 17 6 0 0 0
/proc/31917/statm: 111833 5996 371 364 0 47721 0
[pid=31919] ppid=31917 vsize=3932 CPUtime=374.11 cores=0-7
/proc/31919/stat : 31919 (march_hi) R 31917 31919 31919 0 -1 4202496 1691 0 0 0 37411 0 0 0 25 0 1 0 669168334 4026368 795 33554432000 4194304 4955518 140734970580704 18446744073709551615 4285664 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31919/statm: 983 795 106 186 0 796 0
[pid=31921] ppid=31874 vsize=457588 CPUtime=1.53 cores=0-7
/proc/31921/stat : 31921 (python) S 31874 31874 31027 0 -1 4202560 895 0 0 0 85 68 0 0 15 0 1 0 669168337 468570112 5987 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 17031439516591685895 0 0 17 3 0 0 0
/proc/31921/statm: 114397 5987 371 364 0 50282 0
[pid=31923] ppid=31921 vsize=3932 CPUtime=374.58 cores=0-7
/proc/31923/stat : 31923 (march_hi) R 31921 31923 31923 0 -1 4202496 1699 0 0 0 37458 0 0 0 25 0 1 0 669168338 4026368 798 33554432000 4194304 4955518 140733493985248 18446744073709551615 4314227 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31923/statm: 983 798 106 186 0 796 0
[pid=31924] ppid=31874 vsize=467844 CPUtime=1.52 cores=0-7
/proc/31924/stat : 31924 (python) S 31874 31874 31027 0 -1 4202560 894 0 0 0 79 73 0 0 15 0 1 0 669168342 479072256 6036 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 7436819577641599239 0 0 17 1 0 0 0
/proc/31924/statm: 116961 6036 371 364 0 52843 0
[pid=31927] ppid=31924 vsize=3932 CPUtime=374.18 cores=0-7
/proc/31927/stat : 31927 (march_hi) R 31924 31927 31927 0 -1 4202496 1694 0 0 0 37413 5 0 0 25 0 1 0 669168342 4026368 794 33554432000 4194304 4955518 140734439315648 18446744073709551615 4237810 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31927/statm: 983 794 106 186 0 796 0
[pid=31928] ppid=31874 vsize=478100 CPUtime=1.6 cores=0-7
/proc/31928/stat : 31928 (python) S 31874 31874 31027 0 -1 4202560 24744 0 0 0 81 79 0 0 15 0 1 0 669168346 489574400 6065 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 16278318032030236932 0 0 17 5 0 0 0
/proc/31928/statm: 119525 6065 372 364 0 55404 0
[pid=31930] ppid=31928 vsize=3932 CPUtime=372.9 cores=0-7
/proc/31930/stat : 31930 (march_hi) R 31928 31930 31930 0 -1 4202496 1692 0 0 0 37290 0 0 0 25 0 1 0 669168346 4026368 792 33554432000 4194304 4955518 140735864631616 18446744073709551615 4224450 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31930/statm: 983 792 106 186 0 796 0
[pid=31938] ppid=31874 vsize=488356 CPUtime=0.75 cores=0-7
/proc/31938/stat : 31938 (python) S 31874 31874 31027 0 -1 4202560 985 0 0 0 40 35 0 0 15 0 1 0 669187534 500076544 6037 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 7926445299629785351 0 0 17 5 0 0 0
/proc/31938/statm: 122089 6037 379 364 0 57965 0
[pid=31940] ppid=31938 vsize=3080 CPUtime=182.54 cores=0-7
/proc/31940/stat : 31940 (march_hi) R 31938 31940 31940 0 -1 4202496 1477 0 0 0 18254 0 0 0 25 0 1 0 669187535 3153920 575 33554432000 4194304 4955518 140734311398496 18446744073709551615 4297475 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/31940/statm: 770 575 106 186 0 583 0
[pid=31941] ppid=31874 vsize=498612 CPUtime=0.35 cores=0-7
/proc/31941/stat : 31941 (python) S 31874 31874 31027 0 -1 4202560 1125 0 0 0 18 17 0 0 19 0 2 0 669188032 510578688 6134 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/31941/statm: 124653 6134 407 364 0 60526 0
[pid=31943] ppid=31941 vsize=246940 CPUtime=97.45 cores=0-7
/proc/31943/stat : 31943 (FH) T 31941 31943 31943 0 -1 4202496 602 0 0 0 9745 0 0 0 25 0 1 0 669188032 252866560 326 33554432000 134512640 134936988 4293331776 18446744073709551615 134554090 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/31943/statm: 61735 326 41 104 0 61239 0
[pid=31941/tid=31947] ppid=31874 vsize=498612 CPUtime=0 cores=0-7
/proc/31941/task/31947/stat : 31947 (python) S 31874 31874 31027 0 -1 4202560 78 0 0 0 0 0 0 0 19 0 2 0 669198069 510578688 6134 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 514 18446604462944737408 0 0 -1 0 0 0 0
[pid=31948] ppid=31874 vsize=508868 CPUtime=0.28 cores=0-7
/proc/31948/stat : 31948 (python) S 31874 31874 31027 0 -1 4202560 932 0 0 0 14 14 0 0 18 0 1 0 669198074 521080832 6043 33554432000 4194304 5683644 140733271031792 18446744073709551615 212436037190 0 0 16781312 514 18446744073709519111 0 0 17 7 0 0 0
/proc/31948/statm: 127217 6043 373 364 0 63087 0
[pid=31950] ppid=31948 vsize=268424 CPUtime=76.46 cores=0-7
/proc/31950/stat : 31950 (hybridGM3) R 31948 31950 31950 0 -1 4202496 4885 0 0 0 7646 0 0 0 25 0 1 0 669198074 274866176 4610 33554432000 134512640 135204385 4291716032 18446744073709551615 134537272 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31950/statm: 67106 4610 64 169 0 66936 0
[pid=31874/tid=31881] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31881/stat : 31881 (python) S 31872 31874 31027 0 -1 4202560 26 937 0 0 0 0 0 0 19 0 18 0 669158292 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 0 0 0 0
[pid=31874/tid=31884] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31884/stat : 31884 (python) S 31872 31874 31027 0 -1 4202560 22 937 0 0 0 0 0 0 21 0 18 0 669158295 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 6 0 0 0
[pid=31874/tid=31887] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31887/stat : 31887 (python) S 31872 31874 31027 0 -1 4202560 30 937 0 0 0 0 0 0 18 0 18 0 669158298 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 6 0 0 0
[pid=31874/tid=31890] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31890/stat : 31890 (python) S 31872 31874 31027 0 -1 4202560 30 937 0 0 0 0 0 0 18 0 18 0 669158300 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 5 0 0 0
[pid=31874/tid=31893] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31893/stat : 31893 (python) S 31872 31874 31027 0 -1 4202560 84 937 0 0 0 0 0 0 18 0 18 0 669158304 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 0 0 0 0
[pid=31874/tid=31896] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31896/stat : 31896 (python) S 31872 31874 31027 0 -1 4202560 34 937 0 0 0 0 0 0 21 0 18 0 669158307 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 0 0 0 0
[pid=31874/tid=31899] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31899/stat : 31899 (python) S 31872 31874 31027 0 -1 4202560 35 937 0 0 0 0 0 0 23 0 18 0 669158310 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 4 0 0 0
[pid=31874/tid=31902] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31902/stat : 31902 (python) S 31872 31874 31027 0 -1 4202560 41 937 0 0 0 0 0 0 18 0 18 0 669158313 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 0 0 0 0
[pid=31874/tid=31909] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31909/stat : 31909 (python) S 31872 31874 31027 0 -1 4202560 43 937 0 0 0 0 0 0 20 0 18 0 669168232 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 1 0 0 0
[pid=31874/tid=31913] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31913/stat : 31913 (python) S 31872 31874 31027 0 -1 4202560 35 937 0 0 0 0 0 0 21 0 18 0 669168245 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 7 0 0 0
[pid=31874/tid=31918] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31918/stat : 31918 (python) S 31872 31874 31027 0 -1 4202560 33 937 0 0 0 0 0 0 19 0 18 0 669168333 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 0 0 0 0
[pid=31874/tid=31922] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31922/stat : 31922 (python) S 31872 31874 31027 0 -1 4202560 33 937 0 0 0 0 0 0 20 0 18 0 669168338 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 7 0 0 0
[pid=31874/tid=31926] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31926/stat : 31926 (python) S 31872 31874 31027 0 -1 4202560 31 937 0 0 0 0 0 0 20 0 18 0 669168342 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 1 0 0 0
[pid=31874/tid=31929] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31929/stat : 31929 (python) S 31872 31874 31027 0 -1 4202560 38 937 0 0 0 0 0 0 22 0 18 0 669168346 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 5 0 0 0
[pid=31874/tid=31939] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31939/stat : 31939 (python) S 31872 31874 31027 0 -1 4202560 42 937 0 0 0 0 0 0 18 0 18 0 669187534 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 0 0 0 0
[pid=31874/tid=31942] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31942/stat : 31942 (python) S 31872 31874 31027 0 -1 4202560 33 937 0 0 0 0 0 0 20 0 18 0 669188032 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 7 0 0 0
[pid=31874/tid=31949] ppid=31872 vsize=519112 CPUtime=0 cores=0-7
/proc/31874/task/31949/stat : 31949 (python) S 31872 31874 31027 0 -1 4202560 34 937 0 0 0 0 0 0 20 0 18 0 669198074 531570688 7584 33554432000 4194304 5683644 140733271031792 18446744073709551615 212447840689 0 0 16781312 2 18446604444200766464 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3801.17
Current children cumulated vsize (KiB) 13032720

Child status: 0
Real time (s): 477.492
CPU time (s): 3801.88
CPU user time (s): 3794.46
CPU system time (s): 7.41487
CPU usage (%): 796.217
Max. virtual memory (cumulated for all children) (KiB): 13032720

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3794.46
system time used= 7.41487
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 93639
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 8848
involuntary context switches= 24784

runsolver used 3.61045 second user time and 7.54185 second system time

The end

Launcher Data 

Begin job on node102 at 2011-05-16 21:52:38
IDJOB=3385824
IDBENCH=83089
IDSOLVER=1870
FILE ID=node102/3385824-1305575558
RUNJOBID= node102-1305575311-31041
PBS_JOBID= 13350161
Free space on /tmp= 73580 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k7-r89-v60-c5340-S2117313371-002.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385824-1305575558/watcher-3385824-1305575558 -o /tmp/evaluation-result-3385824-1305575558/solver-3385824-1305575558 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385824-1305575558.cnf -seed 1676964751 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 80ed5c7022fbd1c0bf0af45d87bcb3f8
RANDOM SEED=1676964751

node102.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      10303928 kB
Buffers:        134048 kB
Cached:        1056556 kB
SwapCached:      33496 kB
Active:       13778168 kB
Inactive:      8613800 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      10303928 kB
SwapTotal:    67111528 kB
SwapFree:     56316216 kB
Dirty:           32408 kB
Writeback:           0 kB
AnonPages:    21180084 kB
Mapped:          24240 kB
Slab:            81956 kB
PageTables:      98732 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 134969800 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73548 MiB
End job on node102 at 2011-05-16 22:00:36