Trace number 3378153

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 2417.08 609.433

General information on the benchmark

Namerandom/large/
unif-k5-r20-v1250-c25000-S573815729-001-UNKNOWN.cnf
MD5SUM34a446d3b38e7b034208a35fa75247bf
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark172.11
Satisfiable
(Un)Satisfiability was proved
Number of variables1250
Number of clauses25000
Sum of the clauses size125000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 525000
Number of clauses of size over 50

Solver Data 

0.32/0.46	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.32/0.46	c INFO: solving HOME/instance-3378153-1305463185.cnf
0.32/0.46	c INFO: computing task features
0.42/0.59	c INFO: computed variable-clause graph statistics
0.42/0.60	c INFO: computed clause balance statistics
1.11/1.25	c INFO: computed variable balance statistics
1.51/1.62	c INFO: computed small-clause counts
1.51/1.62	c INFO: computed Horn-clause counts
1.51/1.65	c INFO: computed variable graph statistics
1.51/1.65	c INFO: collected features for HOME/instance-3378153-1305463185.cnf in 1.18 s
1.51/1.65	c INFO: running gnovelty+2@0 for 97 with 1198 remaining (b = 0.38)
1.51/1.66	c INFO: running gnovelty+2@0 for 97 with 1198 remaining (b = 0.14)
1.51/1.67	c INFO: running adaptg2wsat2009++@0 for 97 with 1198 remaining (b = 0.09)
1.51/1.68	c INFO: running adaptg2wsat2009++@0 for 97 with 1198 remaining (b = 0.07)
401.13/101.98	c INFO: running adaptg2wsat2009++@0 for 97 with 1098 remaining (b = 0.06)
401.13/101.99	c INFO: running gnovelty+2@0 for 97 with 1098 remaining (b = 0.05)
401.47/102.01	c INFO: running FH@0 for 97 with 1098 remaining (b = 0.04)
401.59/102.11	c INFO: running adaptg2wsat2009++@0 for 97 with 1098 remaining (b = 0.03)
800.12/202.29	c INFO: running gnovelty+2@0 for 97 with 998 remaining (b = 0.03)
800.53/202.31	c INFO: running gnovelty+2@0 for 97 with 998 remaining (b = 0.03)
800.53/202.33	c INFO: running gnovelty+2@100 for 97 with 998 remaining (b = 0.03)
801.29/202.59	c INFO: running adaptg2wsat2009++@0 for 97 with 997 remaining (b = 0.03)
1187.07/299.59	c INFO: running FH@0 for 97 with 900 remaining (b = 0.03)
1187.47/299.62	c INFO: running adaptg2wsat2009++@0 for 97 with 900 remaining (b = 0.02)
1200.57/302.90	c INFO: running hybridGM3@0 for 194 with 897 remaining (b = 0.04)
1203.40/303.63	c INFO: running gnovelty+2@0 for 97 with 896 remaining (b = 0.02)
1582.28/398.94	c INFO: running hybridGM3@0 for 194 with 801 remaining (b = 0.03)
1586.23/399.96	c INFO: running FH@0 for 97 with 800 remaining (b = 0.01)
1602.52/404.00	c INFO: running hybridGM3@0 for 194 with 796 remaining (b = 0.03)
1981.46/499.55	c INFO: running hybridGM3@98 for 97 with 700 remaining (b = 0.01)
2008.45/506.32	c INFO: running adaptg2wsat2009++@0 for 97 with 694 remaining (b = 0.01)
2358.72/594.61	c INFO: running hybridGM3@100 for 97 with 605 remaining (b = 0.01)
2374.98/598.72	c INFO: running adaptg2wsat2009++@98 for 97 with 601 remaining (b = 0.01)
2383.29/600.87	c INFO: running iPAWS@0 for 97 with 599 remaining (b = 0.01)
2414.23/608.67	c INFO: running hybridGM3@0 for 194 with 591 remaining (b = 0.02)
2417.08/609.36	c INFO: running TNM@0 for 591 with 591 remaining (b = 0.07)
2417.08/609.41	c INFO: running adaptg2wsat2009++@99 for 97 with 591 remaining (b = 0.01)
2417.08/609.41	Traceback (most recent call last):
2417.08/609.41	  File "HOME/solve", line 33, in <module>
2417.08/609.41	    plac.call(main)
2417.08/609.41	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 311, in call
2417.08/609.41	    cmd, result = parser_from(obj).consume(arglist)
2417.08/609.41	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 192, in consume
2417.08/609.41	    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
2417.08/609.41	  File "HOME/borg/build/lib/borg/tools/solve.py", line 84, in main
2417.08/609.41	    (_, answer, _) = solver(input_path, budget / cores, cores)
2417.08/609.41	  File "HOME/borg/build/lib/borg/bilevel.py", line 202, in __call__
2417.08/609.41	    solver.go(max_cost * borg.defaults.machine_speed)
2417.08/609.41	  File "HOME/borg/build/lib/borg/solvers.py", line 179, in go
2417.08/609.41	    self._process.start()
2417.08/609.41	  File "/home/evaluation/evaluation/tools64/lib/python2.7/multiprocessing/process.py", line 94, in start
2417.08/609.42	    assert self._popen is None, 'cannot start a process twice'
2417.08/609.42	AssertionError: cannot start a process twice

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378153-1305463185/watcher-3378153-1305463185 -o /tmp/evaluation-result-3378153-1305463185/solver-3378153-1305463185 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378153-1305463185.cnf -seed 358429141 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.12 5.66 4.47 6/391 2393
/proc/meminfo: memFree=26808052/32950928 swapFree=60199044/67111528
[pid=2393] ppid=2391 vsize=15380 CPUtime=0 cores=0,2,4,6
/proc/2393/stat : 2393 (python) S 2391 2393 2082 0 -1 4202496 809 0 0 0 0 0 0 0 25 0 1 0 657917263 15749120 696 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272687104 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/2393/statm: 3845 696 334 364 0 332 0

[startup+0.0351481 s]
/proc/loadavg: 8.12 5.66 4.47 6/391 2393
/proc/meminfo: memFree=26808052/32950928 swapFree=60199044/67111528
[pid=2393] ppid=2391 vsize=18884 CPUtime=0.02 cores=0,2,4,6
/proc/2393/stat : 2393 (python) R 2391 2393 2082 0 -1 4202496 1284 0 0 0 2 0 0 0 25 0 1 0 657917263 19337216 1138 33554432000 4194304 5683644 140734164333488 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/2393/statm: 4721 1138 421 364 0 690 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 18884

[startup+0.100138 s]
/proc/loadavg: 8.12 5.66 4.47 6/391 2393
/proc/meminfo: memFree=26808052/32950928 swapFree=60199044/67111528
[pid=2393] ppid=2391 vsize=104228 CPUtime=0.06 cores=0,2,4,6
/proc/2393/stat : 2393 (python) R 2391 2393 2082 0 -1 4202496 3064 0 0 0 5 1 0 0 25 0 1 0 657917263 106729472 2747 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272732295 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/2393/statm: 26057 2748 847 364 0 1805 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 104228

[startup+0.300103 s]
/proc/loadavg: 8.12 5.66 4.47 6/391 2393
/proc/meminfo: memFree=26808052/32950928 swapFree=60199044/67111528
[pid=2393] ppid=2391 vsize=190252 CPUtime=0.23 cores=0,2,4,6
/proc/2393/stat : 2393 (python) R 2391 2393 2082 0 -1 4202496 6857 934 1 0 17 6 0 0 25 0 1 0 657917263 194818048 4913 33554432000 4194304 5683644 140734164333488 18446744073709551615 268267719536 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/2393/statm: 47563 4914 1411 364 0 3723 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 190252

[startup+0.701029 s]
/proc/loadavg: 8.12 5.66 4.47 6/391 2393
/proc/meminfo: memFree=26808052/32950928 swapFree=60199044/67111528
[pid=2393] ppid=2391 vsize=285808 CPUtime=0.61 cores=0,2,4,6
/proc/2393/stat : 2393 (python) R 2391 2393 2082 0 -1 4202496 12054 934 1 0 51 10 0 0 25 0 1 0 657917263 292667392 8837 33554432000 4194304 5683644 140734164333488 18446744073709551615 4663081 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/2393/statm: 71452 8837 1826 364 0 7376 0
Current children cumulated CPU time (s) 0.61
Current children cumulated vsize (KiB) 285808

[startup+1.50089 s]
/proc/loadavg: 8.12 5.66 4.47 6/392 2398
/proc/meminfo: memFree=26778364/32950928 swapFree=60199044/67111528
[pid=2393] ppid=2391 vsize=285808 CPUtime=1.41 cores=0,2,4,6
/proc/2393/stat : 2393 (python) R 2391 2393 2082 0 -1 4202496 12055 934 1 0 131 10 0 0 25 0 1 0 657917263 292667392 8838 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272346889 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/2393/statm: 71452 8838 1827 364 0 7376 0
Current children cumulated CPU time (s) 1.41
Current children cumulated vsize (KiB) 285808

[startup+3.10061 s]
/proc/loadavg: 8.12 5.66 4.47 9/404 2410
/proc/meminfo: memFree=26753460/32950928 swapFree=60199044/67111528
[pid=2393] ppid=2391 vsize=323056 CPUtime=1.6 cores=0,2,4,6
/proc/2393/stat : 2393 (python) S 2391 2393 2082 0 -1 4202496 14892 934 1 0 149 11 0 0 25 0 5 0 657917263 330809344 7955 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2393/statm: 80764 7955 1875 364 0 16673 0
[pid=2399] ppid=2393 vsize=282044 CPUtime=0 cores=0,2,4,6
/proc/2399/stat : 2399 (python) S 2393 2393 2082 0 -1 4202560 727 0 0 0 0 0 0 0 18 0 1 0 657917429 288813056 6418 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272709135 0 0 16781312 514 18273544608350699780 0 0 17 6 0 0 0
/proc/2399/statm: 70511 6418 354 364 0 6429 0
[pid=2401] ppid=2399 vsize=1018800 CPUtime=1.43 cores=0,2,4,6
/proc/2401/stat : 2401 (gnovelty+2) R 2399 2401 2401 0 -1 4202496 866 0 0 0 143 0 0 0 25 0 1 0 657917429 1043251200 603 33554432000 134512640 135663704 4288204400 18446744073709551615 134544122 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/2401/statm: 254700 603 113 282 0 254416 0
[pid=2402] ppid=2393 vsize=292300 CPUtime=0 cores=0,2,4,6
/proc/2402/stat : 2402 (python) S 2393 2393 2082 0 -1 4202560 740 0 0 0 0 0 0 0 18 0 1 0 657917430 299315200 6416 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272709190 0 0 16781312 514 18273540760060002564 0 0 17 6 0 0 0
/proc/2402/statm: 73075 6416 348 364 0 8990 0
[pid=2404] ppid=2402 vsize=1018800 CPUtime=1.42 cores=0,2,4,6
/proc/2404/stat : 2404 (gnovelty+2) R 2402 2404 2404 0 -1 4202496 865 0 0 0 142 0 0 0 25 0 1 0 657917430 1043251200 603 33554432000 134512640 135663704 4292368400 18446744073709551615 134525258 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/2404/statm: 254700 603 113 282 0 254416 0
[pid=2405] ppid=2393 vsize=302556 CPUtime=0 cores=0,2,4,6
/proc/2405/stat : 2405 (python) S 2393 2393 2082 0 -1 4202560 737 0 0 0 0 0 0 0 18 0 1 0 657917431 309817344 6420 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272709190 0 0 16781312 514 18273536911769337855 0 0 17 4 0 0 0
/proc/2405/statm: 75639 6420 348 364 0 11551 0
[pid=2407] ppid=2405 vsize=420484 CPUtime=1.39 cores=0,2,4,6
/proc/2407/stat : 2407 (adaptg2wsat2009) R 2405 2407 2407 0 -1 4202496 1756 0 0 0 139 0 0 0 25 0 1 0 657917431 430575616 1493 33554432000 134512640 135107411 4289257952 18446744073709551615 134532416 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/2407/statm: 105121 1493 59 146 0 103997 0
[pid=2408] ppid=2393 vsize=312812 CPUtime=0 cores=0,2,4,6
/proc/2408/stat : 2408 (python) S 2393 2393 2082 0 -1 4202560 743 0 0 0 0 0 0 0 18 0 1 0 657917432 320319488 6424 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272709190 0 0 16781312 514 18273533063478640639 0 0 17 4 0 0 0
/proc/2408/statm: 78203 6424 348 364 0 14112 0
[pid=2410] ppid=2408 vsize=420484 CPUtime=1.4 cores=0,2,4,6
/proc/2410/stat : 2410 (adaptg2wsat2009) R 2408 2410 2410 0 -1 4202496 1756 0 0 0 140 0 0 0 25 0 1 0 657917432 430575616 1493 33554432000 134512640 135107411 4294394688 18446744073709551615 134531830 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/2410/statm: 105121 1493 59 146 0 103997 0
[pid=2393/tid=2400] ppid=2391 vsize=323056 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2400/stat : 2400 (python) S 2391 2393 2082 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 5 0 657917429 330809344 7955 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
[pid=2393/tid=2403] ppid=2391 vsize=323056 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2403/stat : 2403 (python) S 2391 2393 2082 0 -1 4202560 23 934 0 0 0 0 0 0 25 0 5 0 657917430 330809344 7955 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
[pid=2393/tid=2406] ppid=2391 vsize=323056 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2406/stat : 2406 (python) S 2391 2393 2082 0 -1 4202560 25 934 0 0 0 0 0 0 25 0 5 0 657917431 330809344 7955 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
[pid=2393/tid=2409] ppid=2391 vsize=323056 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2409/stat : 2409 (python) S 2391 2393 2082 0 -1 4202560 29 934 0 0 0 0 0 0 25 0 5 0 657917432 330809344 7955 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 7.24
Current children cumulated vsize (KiB) 4391336

[startup+6.30104 s]
/proc/loadavg: 8.11 5.69 4.49 9/404 2410
/proc/meminfo: memFree=26750480/32950928 swapFree=60199044/67111528
[pid=2393] ppid=2391 vsize=323056 CPUtime=1.6 cores=0,2,4,6
/proc/2393/stat : 2393 (python) S 2391 2393 2082 0 -1 4202496 14892 934 1 0 149 11 0 0 25 0 5 0 657917263 330809344 7955 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2393/statm: 80764 7955 1875 364 0 16673 0
[pid=2399] ppid=2393 vsize=282044 CPUtime=0 cores=0,2,4,6
/proc/2399/stat : 2399 (python) S 2393 2393 2082 0 -1 4202560 887 0 0 0 0 0 0 0 18 0 1 0 657917429 288813056 6441 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272709135 0 0 16781312 514 18273544608350699780 0 0 17 6 0 0 0
/proc/2399/statm: 70511 6441 377 364 0 6429 0
[pid=2401] ppid=2399 vsize=1018800 CPUtime=4.63 cores=0,2,4,6
/proc/2401/stat : 2401 (gnovelty+2) R 2399 2401 2401 0 -1 4202496 866 0 0 0 463 0 0 0 25 0 1 0 657917429 1043251200 603 33554432000 134512640 135663704 4288204400 18446744073709551615 134545631 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/2401/statm: 254700 603 113 282 0 254416 0
[pid=2402] ppid=2393 vsize=292300 CPUtime=0.01 cores=0,2,4,6
/proc/2402/stat : 2402 (python) S 2393 2393 2082 0 -1 4202560 913 0 0 0 0 1 0 0 18 0 1 0 657917430 299315200 6439 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272709190 0 0 16781312 514 18273540760060002564 0 0 17 6 0 0 0
/proc/2402/statm: 73075 6439 371 364 0 8990 0
[pid=2404] ppid=2402 vsize=1018800 CPUtime=4.61 cores=0,2,4,6
/proc/2404/stat : 2404 (gnovelty+2) R 2402 2404 2404 0 -1 4202496 865 0 0 0 461 0 0 0 25 0 1 0 657917430 1043251200 603 33554432000 134512640 135663704 4292368400 18446744073709551615 134524894 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/2404/statm: 254700 603 113 282 0 254416 0
[pid=2405] ppid=2393 vsize=302556 CPUtime=0.01 cores=0,2,4,6
/proc/2405/stat : 2405 (python) S 2393 2393 2082 0 -1 4202560 902 0 0 0 1 0 0 0 18 0 1 0 657917431 309817344 6443 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272709190 0 0 16781312 514 18273536911769305347 0 0 17 2 0 0 0
/proc/2405/statm: 75639 6443 371 364 0 11551 0
[pid=2407] ppid=2405 vsize=420484 CPUtime=4.52 cores=0,2,4,6
/proc/2407/stat : 2407 (adaptg2wsat2009) R 2405 2407 2407 0 -1 4202496 1756 0 0 0 452 0 0 0 25 0 1 0 657917431 430575616 1493 33554432000 134512640 135107411 4289257952 18446744073709551615 134531239 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/2407/statm: 105121 1493 59 146 0 103997 0
[pid=2408] ppid=2393 vsize=312812 CPUtime=0 cores=0,2,4,6
/proc/2408/stat : 2408 (python) S 2393 2393 2082 0 -1 4202560 919 0 0 0 0 0 0 0 18 0 1 0 657917432 320319488 6447 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272709190 0 0 16781312 514 18273533063478608131 0 0 17 4 0 0 0
/proc/2408/statm: 78203 6447 371 364 0 14112 0
[pid=2410] ppid=2408 vsize=420484 CPUtime=4.57 cores=0,2,4,6
/proc/2410/stat : 2410 (adaptg2wsat2009) R 2408 2410 2410 0 -1 4202496 1756 0 0 0 457 0 0 0 25 0 1 0 657917432 430575616 1493 33554432000 134512640 135107411 4294394688 18446744073709551615 134531731 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/2410/statm: 105121 1493 59 146 0 103997 0
[pid=2393/tid=2400] ppid=2391 vsize=323056 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2400/stat : 2400 (python) S 2391 2393 2082 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 5 0 657917429 330809344 7955 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
[pid=2393/tid=2403] ppid=2391 vsize=323056 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2403/stat : 2403 (python) S 2391 2393 2082 0 -1 4202560 23 934 0 0 0 0 0 0 25 0 5 0 657917430 330809344 7955 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
[pid=2393/tid=2406] ppid=2391 vsize=323056 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2406/stat : 2406 (python) S 2391 2393 2082 0 -1 4202560 25 934 0 0 0 0 0 0 25 0 5 0 657917431 330809344 7955 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
[pid=2393/tid=2409] ppid=2391 vsize=323056 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2409/stat : 2409 (python) S 2391 2393 2082 0 -1 4202560 29 934 0 0 0 0 0 0 25 0 5 0 657917432 330809344 7955 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 19.95
Current children cumulated vsize (KiB) 4391336

[startup+12.7009 s]
/proc/loadavg: 8.10 5.73 4.51 9/404 2411
/proc/meminfo: memFree=26748524/32950928 swapFree=60199044/67111528
[pid=2393] ppid=2391 vsize=323056 CPUtime=1.6 cores=0,2,4,6
/proc/2393/stat : 2393 (python) S 2391 2393 2082 0 -1 4202496 14892 934 1 0 149 11 0 0 25 0 5 0 657917263 330809344 7955 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2393/statm: 80764 7955 1875 364 0 16673 0
[pid=2399] ppid=2393 vsize=282044 CPUtime=0.02 cores=0,2,4,6
/proc/2399/stat : 2399 (python) S 2393 2393 2082 0 -1 4202560 890 0 0 0 1 1 0 0 18 0 1 0 657917429 288813056 6441 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272709135 0 0 16781312 514 15283176441714278660 0 0 17 6 0 0 0
/proc/2399/statm: 70511 6441 377 364 0 6429 0
[pid=2401] ppid=2399 vsize=1018800 CPUtime=11.03 cores=0,2,4,6
/proc/2401/stat : 2401 (gnovelty+2) R 2399 2401 2401 0 -1 4202496 866 0 0 0 1103 0 0 0 25 0 1 0 657917429 1043251200 603 33554432000 134512640 135663704 4288204400 18446744073709551615 134524455 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/2401/statm: 254700 603 113 282 0 254416 0
[pid=2402] ppid=2393 vsize=292300 CPUtime=0.02 cores=0,2,4,6
/proc/2402/stat : 2402 (python) S 2393 2393 2082 0 -1 4202560 916 0 0 0 1 1 0 0 18 0 1 0 657917430 299315200 6439 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272709190 0 0 16781312 514 15283176441714278660 0 0 17 6 0 0 0
/proc/2402/statm: 73075 6439 371 364 0 8990 0
[pid=2404] ppid=2402 vsize=1018800 CPUtime=11 cores=0,2,4,6
/proc/2404/stat : 2404 (gnovelty+2) R 2402 2404 2404 0 -1 4202496 865 0 0 0 1100 0 0 0 25 0 1 0 657917430 1043251200 603 33554432000 134512640 135663704 4292368400 18446744073709551615 134524897 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/2404/statm: 254700 603 113 282 0 254416 0
[pid=2405] ppid=2393 vsize=302556 CPUtime=0.02 cores=0,2,4,6
/proc/2405/stat : 2405 (python) S 2393 2393 2082 0 -1 4202560 913 0 0 0 2 0 0 0 18 0 1 0 657917431 309817344 6443 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272709190 0 0 16781312 514 15283176441714278659 0 0 17 2 0 0 0
/proc/2405/statm: 75639 6443 371 364 0 11551 0
[pid=2407] ppid=2405 vsize=420484 CPUtime=10.8 cores=0,2,4,6
/proc/2407/stat : 2407 (adaptg2wsat2009) R 2405 2407 2407 0 -1 4202496 1756 0 0 0 1080 0 0 0 25 0 1 0 657917431 430575616 1493 33554432000 134512640 135107411 4289257952 18446744073709551615 134532036 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/2407/statm: 105121 1493 59 146 0 103997 0
[pid=2408] ppid=2393 vsize=312812 CPUtime=0.02 cores=0,2,4,6
/proc/2408/stat : 2408 (python) S 2393 2393 2082 0 -1 4202560 921 0 0 0 1 1 0 0 18 0 1 0 657917432 320319488 6447 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272709190 0 0 16781312 514 15283176441714278659 0 0 17 4 0 0 0
/proc/2408/statm: 78203 6447 371 364 0 14112 0
[pid=2410] ppid=2408 vsize=420484 CPUtime=10.95 cores=0,2,4,6
/proc/2410/stat : 2410 (adaptg2wsat2009) R 2408 2410 2410 0 -1 4202496 1756 0 0 0 1095 0 0 0 25 0 1 0 657917432 430575616 1493 33554432000 134512640 135107411 4294394688 18446744073709551615 134531830 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/2410/statm: 105121 1493 59 146 0 103997 0
[pid=2393/tid=2400] ppid=2391 vsize=323056 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2400/stat : 2400 (python) S 2391 2393 2082 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 5 0 657917429 330809344 7955 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
[pid=2393/tid=2403] ppid=2391 vsize=323056 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2403/stat : 2403 (python) S 2391 2393 2082 0 -1 4202560 23 934 0 0 0 0 0 0 25 0 5 0 657917430 330809344 7955 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
[pid=2393/tid=2406] ppid=2391 vsize=323056 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2406/stat : 2406 (python) S 2391 2393 2082 0 -1 4202560 25 934 0 0 0 0 0 0 25 0 5 0 657917431 330809344 7955 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
[pid=2393/tid=2409] ppid=2391 vsize=323056 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2409/stat : 2409 (python) S 2391 2393 2082 0 -1 4202560 29 934 0 0 0 0 0 0 25 0 5 0 657917432 330809344 7955 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 45.46
Current children cumulated vsize (KiB) 4391336

[startup+25.5006 s]
/proc/loadavg: 8.08 5.84 4.56 9/404 2411
/proc/meminfo: memFree=26749248/32950928 swapFree=60199044/67111528
[pid=2393] ppid=2391 vsize=323056 CPUtime=1.6 cores=0,2,4,6

################
# More data... #
################

[pid=2405/tid=2430] ppid=2393 vsize=302556 CPUtime=0 cores=0,2,4,6
/proc/2405/task/2430/stat : 2430 (python) S 2393 2393 2082 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 657927461 309817344 6473 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 514 18446604452872365312 0 0 -1 2 0 0 0
[pid=2408] ppid=2393 vsize=312812 CPUtime=0.32 cores=0,2,4,6
/proc/2408/stat : 2408 (python) S 2393 2393 2082 0 -1 4202560 1111 0 0 0 19 13 0 0 18 0 2 0 657917432 320319488 6477 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272709190 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/2408/statm: 78203 6477 400 364 0 14112 0
[pid=2410] ppid=2408 vsize=420484 CPUtime=110.51 cores=0,2,4,6
/proc/2410/stat : 2410 (adaptg2wsat2009) R 2408 2410 2410 0 -1 4202496 1756 0 0 0 11051 0 0 0 25 0 1 0 657917432 430575616 1493 33554432000 134512640 135107411 4294394688 18446744073709551615 134531768 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/2410/statm: 105121 1493 59 146 0 103997 0
[pid=2408/tid=2431] ppid=2393 vsize=312812 CPUtime=0 cores=0,2,4,6
/proc/2408/task/2431/stat : 2431 (python) S 2393 2393 2082 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 657927461 320319488 6477 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 514 18446604452872364416 0 0 -1 0 0 0 0
[pid=2433] ppid=2393 vsize=323068 CPUtime=0.3 cores=0,2,4,6
/proc/2433/stat : 2433 (python) S 2393 2393 2082 0 -1 4202560 1094 0 0 0 14 16 0 0 19 0 2 0 657927462 330821632 6481 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/2433/statm: 80767 6481 400 364 0 16673 0
[pid=2435] ppid=2433 vsize=420484 CPUtime=100.28 cores=0,2,4,6
/proc/2435/stat : 2435 (adaptg2wsat2009) T 2433 2435 2435 0 -1 4202496 1755 0 0 0 10028 0 0 0 25 0 1 0 657927462 430575616 1492 33554432000 134512640 135107411 4292155056 18446744073709551615 134531731 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/2435/statm: 105121 1492 59 146 0 103997 0
[pid=2433/tid=2461] ppid=2393 vsize=323068 CPUtime=0 cores=0,2,4,6
/proc/2433/task/2461/stat : 2461 (python) S 2393 2393 2082 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 657937491 330821632 6481 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 514 18446604451792811328 0 0 -1 0 0 0 0
[pid=2436] ppid=2393 vsize=333476 CPUtime=0.6 cores=0,2,4,6
/proc/2436/stat : 2436 (python) S 2393 2393 2082 0 -1 4202560 1138 0 0 0 33 27 0 0 18 0 2 0 657927463 341479424 6523 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/2436/statm: 83369 6523 400 364 0 19272 0
[pid=2438] ppid=2436 vsize=1018800 CPUtime=198.38 cores=0,2,4,6
/proc/2438/stat : 2438 (gnovelty+2) T 2436 2438 2438 0 -1 4202496 859 0 0 0 19838 0 0 0 25 0 1 0 657927463 1043251200 602 33554432000 134512640 135663704 4289659728 18446744073709551615 134524649 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/2438/statm: 254700 602 113 282 0 254416 0
[pid=2436/tid=2463] ppid=2393 vsize=333476 CPUtime=0 cores=0,2,4,6
/proc/2436/task/2463/stat : 2463 (python) S 2393 2393 2082 0 -1 4202560 48 0 0 0 0 0 0 0 18 0 2 0 657937495 341479424 6523 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 514 18446604451283784000 0 0 -1 2 0 0 0
[pid=2439] ppid=2393 vsize=343732 CPUtime=0.3 cores=0,2,4,6
/proc/2439/stat : 2439 (python) S 2393 2393 2082 0 -1 4202560 1103 0 0 0 16 14 0 0 19 0 2 0 657927464 351981568 6547 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/2439/statm: 85933 6547 400 364 0 21833 0
[pid=2441] ppid=2439 vsize=250364 CPUtime=98.15 cores=0,2,4,6
/proc/2441/stat : 2441 (FH) T 2439 2441 2441 0 -1 4202496 1668 0 0 0 9814 1 0 0 25 0 1 0 657927464 256372736 1407 33554432000 134512640 134936988 4287312304 18446744073709551615 134554090 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/2441/statm: 62591 1407 41 104 0 62095 0
[pid=2439/tid=2462] ppid=2393 vsize=343732 CPUtime=0 cores=0,2,4,6
/proc/2439/task/2462/stat : 2462 (python) S 2393 2393 2082 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 657937494 351981568 6547 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 514 18446604451283786688 0 0 -1 6 0 0 0
[pid=2443] ppid=2393 vsize=354160 CPUtime=0.3 cores=0,2,4,6
/proc/2443/stat : 2443 (python) S 2393 2393 2082 0 -1 4202560 1151 0 0 0 17 13 0 0 19 0 2 0 657927474 362659840 6580 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2443/statm: 88540 6580 406 364 0 24437 0
[pid=2445] ppid=2443 vsize=420484 CPUtime=99.55 cores=0,2,4,6
/proc/2445/stat : 2445 (adaptg2wsat2009) T 2443 2445 2445 0 -1 4202496 1759 0 0 0 9955 0 0 0 25 0 1 0 657927474 430575616 1491 33554432000 134512640 135107411 4291900560 18446744073709551615 134531508 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/2445/statm: 105121 1491 59 146 0 103997 0
[pid=2443/tid=2464] ppid=2393 vsize=354160 CPUtime=0 cores=0,2,4,6
/proc/2443/task/2464/stat : 2464 (python) S 2393 2393 2082 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657937522 362659840 6580 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 514 18446604452999437056 0 0 -1 6 0 0 0
[pid=2465] ppid=2393 vsize=364616 CPUtime=0.3 cores=0,2,4,6
/proc/2465/stat : 2465 (python) S 2393 2393 2082 0 -1 4202560 1154 0 0 0 16 14 0 0 19 0 2 0 657937523 373366784 6634 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/2465/statm: 91154 6634 406 364 0 27048 0
[pid=2467] ppid=2465 vsize=420484 CPUtime=99.71 cores=0,2,4,6
/proc/2467/stat : 2467 (adaptg2wsat2009) T 2465 2467 2467 0 -1 4202496 1762 0 0 0 9971 0 0 0 25 0 1 0 657937523 430575616 1491 33554432000 134512640 135107411 4289631376 18446744073709551615 134531781 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/2467/statm: 105121 1491 59 146 0 103997 0
[pid=2465/tid=2481] ppid=2393 vsize=364616 CPUtime=0 cores=0,2,4,6
/proc/2465/task/2481/stat : 2481 (python) S 2393 2393 2082 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 657947552 373366784 6634 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 514 18446604446387016128 0 0 -1 2 0 0 0
[pid=2475] ppid=2393 vsize=374872 CPUtime=0.3 cores=0,2,4,6
/proc/2475/stat : 2475 (python) S 2393 2393 2082 0 -1 4202560 1125 0 0 0 16 14 0 0 19 0 2 0 657947223 383868928 6632 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/2475/statm: 93718 6632 400 364 0 29609 0
[pid=2477] ppid=2475 vsize=250372 CPUtime=99.18 cores=0,2,4,6
/proc/2477/stat : 2477 (FH) T 2475 2477 2477 0 -1 4202496 1674 0 0 0 9918 0 0 0 25 0 1 0 657947223 256380928 1408 33554432000 134512640 134936988 4293273536 18446744073709551615 134558720 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/2477/statm: 62593 1408 41 104 0 62097 0
[pid=2475/tid=2590] ppid=2393 vsize=374872 CPUtime=0 cores=0,2,4,6
/proc/2475/task/2590/stat : 2590 (python) S 2393 2393 2082 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 657957156 383868928 6632 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 514 18446604447629283456 0 0 -1 4 0 0 0
[pid=2478] ppid=2393 vsize=385128 CPUtime=0.3 cores=0,2,4,6
/proc/2478/stat : 2478 (python) S 2393 2393 2082 0 -1 4202560 1130 0 0 0 15 15 0 0 19 0 2 0 657947226 394371072 6638 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/2478/statm: 96282 6638 400 364 0 32170 0
[pid=2480] ppid=2478 vsize=420484 CPUtime=100.3 cores=0,2,4,6
/proc/2480/stat : 2480 (adaptg2wsat2009) T 2478 2480 2480 0 -1 4202496 1764 0 0 0 10030 0 0 0 25 0 1 0 657947226 430575616 1492 33554432000 134512640 135107411 4290623280 18446744073709551615 134531584 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/2480/statm: 105121 1492 59 146 0 103997 0
[pid=2478/tid=2594] ppid=2393 vsize=385128 CPUtime=0 cores=0,2,4,6
/proc/2478/task/2594/stat : 2594 (python) S 2393 2393 2082 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657957258 394371072 6638 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 514 18446604449208995904 0 0 -1 2 0 0 0
[pid=2482] ppid=2393 vsize=395384 CPUtime=0.92 cores=0,2,4,6
/proc/2482/stat : 2482 (python) S 2393 2393 2082 0 -1 4202560 1195 0 0 0 49 43 0 0 18 0 2 0 657947554 404873216 6672 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/2482/statm: 98846 6672 400 364 0 34731 0
[pid=2484] ppid=2482 vsize=268420 CPUtime=295.26 cores=0,2,4,6
/proc/2484/stat : 2484 (hybridGM3) T 2482 2484 2484 0 -1 4202496 5625 0 0 0 29525 1 0 0 25 0 1 0 657947554 274862080 5357 33554432000 134512640 135204385 4291509680 18446744073709551615 134538395 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/2484/statm: 67105 5357 63 169 0 66935 0
[pid=2482/tid=2621] ppid=2393 vsize=395384 CPUtime=0 cores=0,2,4,6
/proc/2482/task/2621/stat : 2621 (python) S 2393 2393 2082 0 -1 4202560 48 0 0 0 0 0 0 0 18 0 2 0 657967217 404873216 6672 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 514 18446604449208996800 0 0 -1 2 0 0 0
[pid=2485] ppid=2393 vsize=405896 CPUtime=0.31 cores=0,2,4,6
/proc/2485/stat : 2485 (python) S 2393 2393 2082 0 -1 4202560 1152 0 0 0 15 16 0 0 19 0 2 0 657947626 415637504 6708 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/2485/statm: 101474 6708 400 364 0 37356 0
[pid=2487] ppid=2485 vsize=1018800 CPUtime=99.32 cores=0,2,4,6
/proc/2487/stat : 2487 (gnovelty+2) T 2485 2487 2487 0 -1 4202496 872 0 0 0 9930 2 0 0 25 0 1 0 657947627 1043251200 602 33554432000 134512640 135663704 4289991296 18446744073709551615 134564710 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/2487/statm: 254700 602 113 282 0 254416 0
[pid=2485/tid=2598] ppid=2393 vsize=405896 CPUtime=0 cores=0,2,4,6
/proc/2485/task/2598/stat : 2598 (python) S 2393 2393 2082 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 657957661 415637504 6708 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 514 18446604452999435264 0 0 -1 2 0 0 0
[pid=2591] ppid=2393 vsize=416152 CPUtime=0.66 cores=0,2,4,6
/proc/2591/stat : 2591 (python) S 2393 2393 2082 0 -1 4202560 1200 0 0 0 35 31 0 0 18 0 2 0 657957158 426139648 6712 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272709190 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/2591/statm: 104038 6712 400 364 0 39917 0
[pid=2593] ppid=2591 vsize=268428 CPUtime=208.58 cores=0,2,4,6
/proc/2593/stat : 2593 (hybridGM3) R 2591 2593 2593 0 -1 4202496 5628 0 0 0 20857 1 0 0 25 0 1 0 657957158 274870272 5359 33554432000 134512640 135204385 4293232416 18446744073709551615 134538549 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/2593/statm: 67107 5359 63 169 0 66937 0
[pid=2591/tid=2641] ppid=2393 vsize=416152 CPUtime=0 cores=0,2,4,6
/proc/2591/task/2641/stat : 2641 (python) S 2393 2393 2082 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657976724 426139648 6712 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 514 18446604446387017024 0 0 -1 2 0 0 0
[pid=2595] ppid=2393 vsize=426600 CPUtime=0.32 cores=0,2,4,6
/proc/2595/stat : 2595 (python) S 2393 2393 2082 0 -1 4202560 1214 0 0 0 18 14 0 0 19 0 2 0 657957260 436838400 6771 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2595/statm: 106650 6771 407 364 0 42526 0
[pid=2597] ppid=2595 vsize=250372 CPUtime=102.68 cores=0,2,4,6
/proc/2597/stat : 2597 (FH) T 2595 2597 2597 0 -1 4202496 1680 0 0 0 10267 1 0 0 25 0 1 0 657957260 256380928 1407 33554432000 134512640 134936988 4289317184 18446744073709551615 134552486 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/2597/statm: 62593 1407 41 104 0 62097 0
[pid=2595/tid=2622] ppid=2393 vsize=426600 CPUtime=0 cores=0,2,4,6
/proc/2595/task/2622/stat : 2622 (python) S 2393 2393 2082 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 657967894 436838400 6771 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 514 18446604447629285248 0 0 -1 4 0 0 0
[pid=2599] ppid=2393 vsize=436856 CPUtime=0.62 cores=0,2,4,6
/proc/2599/stat : 2599 (python) S 2393 2393 2082 0 -1 4202560 1236 0 0 0 35 27 0 0 19 0 2 0 657957664 447340544 6776 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2599/statm: 109214 6776 408 364 0 45087 0
[pid=2601] ppid=2599 vsize=268428 CPUtime=194.69 cores=0,2,4,6
/proc/2601/stat : 2601 (hybridGM3) T 2599 2601 2601 0 -1 4202496 5631 0 0 0 19468 1 0 0 25 0 1 0 657957664 274870272 5359 33554432000 134512640 135204385 4290430432 18446744073709551615 134538444 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/2601/statm: 67107 5359 63 169 0 66937 0
[pid=2599/tid=2642] ppid=2393 vsize=436856 CPUtime=0 cores=0,2,4,6
/proc/2599/task/2642/stat : 2642 (python) S 2393 2393 2082 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657977134 447340544 6776 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 514 18446604453895319744 0 0 -1 6 0 0 0
[pid=2643] ppid=2393 vsize=447112 CPUtime=0.02 cores=0,2,4,6
/proc/2643/stat : 2643 (python) S 2393 2393 2082 0 -1 4202560 1031 0 0 0 1 1 0 0 18 0 1 0 657977350 457842688 6765 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272709190 0 0 16781312 514 5552762972738126082 0 0 17 6 0 0 0
/proc/2643/statm: 111778 6765 379 364 0 47648 0
[pid=2645] ppid=2643 vsize=335996 CPUtime=8.17 cores=0,2,4,6
/proc/2645/stat : 2645 (iPAWS) R 2643 2645 2645 0 -1 4202496 1717 0 0 0 817 0 0 0 25 0 1 0 657977351 344059904 1436 33554432000 134512640 135114423 4292242944 18446744073709551615 134517518 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/2645/statm: 83999 1436 59 147 0 83851 0
[pid=2646] ppid=2393 vsize=457368 CPUtime=0 cores=0,2,4,6
/proc/2646/stat : 2646 (python) S 2393 2393 2082 0 -1 4202560 845 0 0 0 0 0 0 0 19 0 1 0 657978131 468344832 6739 33554432000 4194304 5683644 140734164333488 18446744073709551615 268272709190 0 0 16781312 514 5552766821028855807 0 0 17 6 0 0 0
/proc/2646/statm: 114342 6739 349 364 0 50209 0
[pid=2648] ppid=2646 vsize=268428 CPUtime=0.64 cores=0,2,4,6
/proc/2648/stat : 2648 (hybridGM3) R 2646 2648 2648 0 -1 4202496 5633 0 0 0 64 0 0 0 22 0 1 0 657978131 274870272 5359 33554432000 134512640 135204385 4290770544 18446744073709551615 134538339 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/2648/statm: 67107 5359 63 169 0 66937 0
[pid=2393/tid=2400] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2400/stat : 2400 (python) S 2391 2393 2082 0 -1 4202560 28 934 0 0 0 0 0 0 18 0 19 0 657917429 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
[pid=2393/tid=2403] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2403/stat : 2403 (python) S 2391 2393 2082 0 -1 4202560 28 934 0 0 0 0 0 0 18 0 19 0 657917430 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
[pid=2393/tid=2406] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2406/stat : 2406 (python) S 2391 2393 2082 0 -1 4202560 31 934 0 0 0 0 0 0 18 0 19 0 657917431 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
[pid=2393/tid=2409] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2409/stat : 2409 (python) S 2391 2393 2082 0 -1 4202560 33 934 0 0 0 0 0 0 18 0 19 0 657917432 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 4 0 0 0
[pid=2393/tid=2434] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2434/stat : 2434 (python) S 2391 2393 2082 0 -1 4202560 24 934 0 0 0 0 0 0 20 0 19 0 657927462 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 4 0 0 0
[pid=2393/tid=2437] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2437/stat : 2437 (python) S 2391 2393 2082 0 -1 4202560 30 934 0 0 0 0 0 0 18 0 19 0 657927463 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 2 0 0 0
[pid=2393/tid=2440] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2440/stat : 2440 (python) S 2391 2393 2082 0 -1 4202560 28 934 0 0 0 0 0 0 24 0 19 0 657927464 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 4 0 0 0
[pid=2393/tid=2444] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2444/stat : 2444 (python) S 2391 2393 2082 0 -1 4202560 26 934 0 0 0 0 0 0 25 0 19 0 657927474 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 4 0 0 0
[pid=2393/tid=2466] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2466/stat : 2466 (python) S 2391 2393 2082 0 -1 4202560 22 934 0 0 0 0 0 0 20 0 19 0 657937523 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 2 0 0 0
[pid=2393/tid=2476] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2476/stat : 2476 (python) S 2391 2393 2082 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 19 0 657947223 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 2 0 0 0
[pid=2393/tid=2479] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2479/stat : 2479 (python) S 2391 2393 2082 0 -1 4202560 25 934 0 0 0 0 0 0 21 0 19 0 657947226 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 2 0 0 0
[pid=2393/tid=2483] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2483/stat : 2483 (python) S 2391 2393 2082 0 -1 4202560 29 934 0 0 0 0 0 0 18 0 19 0 657947554 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 2 0 0 0
[pid=2393/tid=2486] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2486/stat : 2486 (python) S 2391 2393 2082 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 19 0 657947627 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
[pid=2393/tid=2592] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2592/stat : 2592 (python) S 2391 2393 2082 0 -1 4202560 32 934 0 0 0 0 0 0 18 0 19 0 657957158 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
[pid=2393/tid=2596] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2596/stat : 2596 (python) S 2391 2393 2082 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 657957260 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
[pid=2393/tid=2600] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2600/stat : 2600 (python) S 2391 2393 2082 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 19 0 657957664 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
[pid=2393/tid=2644] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2644/stat : 2644 (python) S 2391 2393 2082 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 19 0 657977351 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
[pid=2393/tid=2647] ppid=2391 vsize=467612 CPUtime=0 cores=0,2,4,6
/proc/2393/task/2647/stat : 2647 (python) S 2391 2393 2082 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 19 0 657978131 478834688 8306 33554432000 4194304 5683644 140734164333488 18446744073709551615 268284512689 0 0 16781312 2 18446604452999434368 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2417.08
Current children cumulated vsize (KiB) 15890900

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 2393 and gives
#  childrusage.ru_utime.tv_sec=333
#  childrusage.ru_utime.tv_usec=214343
#  childrusage.ru_stime.tv_sec=3
#  childrusage.ru_stime.tv_usec=723433
# CPU time returned by wait4() is 336.938
# while last known CPU time is 2417.08
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 609.433
CPU time (s): 2417.08
CPU user time (s): 2413.42
CPU system time (s): 3.66
CPU usage (%): 396.611
Max. virtual memory (cumulated for all children) (KiB): 15890900

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 333.214
system time used= 3.72343
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 59746
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 5110
involuntary context switches= 2065

runsolver used 5.12622 second user time and 9.52955 second system time

The end

Launcher Data 

Begin job on node122 at 2011-05-15 14:39:45
IDJOB=3378153
IDBENCH=82562
IDSOLVER=1859
FILE ID=node122/3378153-1305463185
RUNJOBID= node122-1305462946-2100
PBS_JOBID= 13324977
Free space on /tmp= 73428 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k5-r20-v1250-c25000-S573815729-001-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378153-1305463185/watcher-3378153-1305463185 -o /tmp/evaluation-result-3378153-1305463185/solver-3378153-1305463185 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378153-1305463185.cnf -seed 358429141 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 34a446d3b38e7b034208a35fa75247bf
RANDOM SEED=358429141

node122.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      26808208 kB
Buffers:        309668 kB
Cached:        3918536 kB
SwapCached:      50104 kB
Active:        1782480 kB
Inactive:      4119880 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      26808208 kB
SwapTotal:    67111528 kB
SwapFree:     60199044 kB
Dirty:           32548 kB
Writeback:           0 kB
AnonPages:     1634732 kB
Mapped:          25716 kB
Slab:           130768 kB
PageTables:      43904 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 85203248 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73388 MiB
End job on node122 at 2011-05-15 14:49:56