Trace number 3378219

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? 3484.35 1139.13

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v25000-c105000-S30704505-092-UNKNOWN.cnf
MD5SUM15b86e0c1384321ea7bbac2466a811a7
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark318.072
Satisfiable
(Un)Satisfiability was proved
Number of variables25000
Number of clauses105000
Sum of the clauses size315000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3105000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.37/0.63	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.37/0.63	c INFO: solving HOME/instance-3378219-1305463843.cnf
0.37/0.63	c INFO: computing task features
0.66/1.00	c INFO: computed variable-clause graph statistics
0.66/1.01	c INFO: computed clause balance statistics
2.55/2.82	c INFO: computed variable balance statistics
3.95/4.30	c INFO: computed small-clause counts
3.95/4.31	c INFO: computed Horn-clause counts
4.85/5.14	c INFO: computed variable graph statistics
4.85/5.14	c INFO: collected features for HOME/instance-3378219-1305463843.cnf in 4.50 s
4.85/5.15	c INFO: running hybridGM3@0 for 97 with 1195 remaining (b = 0.40)
4.85/5.16	c INFO: running hybridGM3@0 for 97 with 1195 remaining (b = 0.25)
4.85/5.17	c INFO: running hybridGM3@0 for 97 with 1195 remaining (b = 0.17)
4.85/5.18	c INFO: running hybridGM3@0 for 194 with 1195 remaining (b = 0.23)
372.37/127.61	c INFO: running hybridGM3@0 for 97 with 1073 remaining (b = 0.10)
428.48/146.33	c INFO: running hybridGM3@0 for 97 with 1054 remaining (b = 0.10)
451.48/154.08	c INFO: running hybridGM3@0 for 97 with 1046 remaining (b = 0.10)
686.39/232.04	c INFO: running hybridGM3@100 for 97 with 968 remaining (b = 0.10)
774.10/261.45	c INFO: running hybridGM3@92 for 97 with 939 remaining (b = 0.10)
813.80/274.71	c INFO: running hybridGM3@102 for 97 with 925 remaining (b = 0.10)
886.01/298.90	c INFO: running hybridGM3@99 for 97 with 901 remaining (b = 0.10)
1003.96/338.42	c INFO: running TNM@0 for 97 with 862 remaining (b = 0.12)
1167.44/393.24	c INFO: running hybridGM3@0 for 194 with 807 remaining (b = 0.17)
1277.81/430.24	c INFO: running hybridGM3@0 for 97 with 770 remaining (b = 0.07)
1310.05/441.06	c INFO: running hybridGM3@0 for 194 with 759 remaining (b = 0.10)
1321.32/444.80	c INFO: running hybridGM3@0 for 194 with 755 remaining (b = 0.07)
1702.80/561.03	c INFO: running hybridGM3@95 for 97 with 639 remaining (b = 0.03)
1987.85/656.46	c INFO: running hybridGM3@95 for 97 with 544 remaining (b = 0.03)
2008.28/662.39	c INFO: running hybridGM3@0 for 194 with 538 remaining (b = 0.06)
2127.48/702.01	c INFO: running hybridGM3@102 for 97 with 498 remaining (b = 0.03)
2188.08/722.27	c INFO: running hybridGM3@99 for 97 with 478 remaining (b = 0.03)
2294.94/757.82	c INFO: running hybridGM3@191 for 97 with 442 remaining (b = 0.03)
2521.67/833.48	c INFO: running TNM@93 for 367 with 367 remaining (b = 0.07)
2529.32/835.67	c INFO: running gnovelty+2@0 for 97 with 364 remaining (b = 0.03)
2785.75/919.39	c INFO: running hybridGM3@293 for 97 with 281 remaining (b = 0.02)
2894.22/955.69	c INFO: running hybridGM3@299 for 97 with 244 remaining (b = 0.02)
2998.24/990.44	c INFO: running TNM@0 for 97 with 210 remaining (b = 0.02)
3076.83/1016.76	c INFO: running TNM@0 for 183 with 183 remaining (b = 0.02)
3257.63/1071.12	c INFO: running hybridGM3@0 for 129 with 129 remaining (b = 0.02)
3484.27/1139.10	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378219-1305463843/watcher-3378219-1305463843 -o /tmp/evaluation-result-3378219-1305463843/solver-3378219-1305463843 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378219-1305463843.cnf -seed 1835479921 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 10.27 9.77 7.69 7/498 17454
/proc/meminfo: memFree=24795884/32951124 swapFree=53501244/67111528
[pid=17454] ppid=17452 vsize=16240 CPUtime=0 cores=1,3,5,7
/proc/17454/stat : 17454 (python) R 17452 17454 16982 0 -1 4202496 1015 0 0 0 0 0 0 0 25 0 1 0 657986928 16629760 890 33554432000 4194304 5683644 140736153411008 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/17454/statm: 4060 893 372 364 0 547 0

[startup+0.0472581 s]
/proc/loadavg: 10.27 9.77 7.69 7/498 17454
/proc/meminfo: memFree=24795884/32951124 swapFree=53501244/67111528
[pid=17454] ppid=17452 vsize=18880 CPUtime=0.02 cores=1,3,5,7
/proc/17454/stat : 17454 (python) R 17452 17454 16982 0 -1 4202496 1286 0 0 0 2 0 0 0 25 0 1 0 657986928 19333120 1140 33554432000 4194304 5683644 140736153411008 18446744073709551615 4298734 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/17454/statm: 4720 1140 421 364 0 689 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 18880

[startup+0.100247 s]
/proc/loadavg: 10.27 9.77 7.69 7/498 17454
/proc/meminfo: memFree=24795884/32951124 swapFree=53501244/67111528
[pid=17454] ppid=17452 vsize=93956 CPUtime=0.07 cores=1,3,5,7
/proc/17454/stat : 17454 (python) S 17452 17454 16982 0 -1 4202496 2690 0 0 0 5 2 0 0 17 0 1 0 657986928 96210944 2406 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634513408 0 2147483391 16781312 2 18446744071699199166 0 0 17 3 0 0 0
/proc/17454/statm: 23489 2406 764 364 0 1621 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 93956

[startup+0.30023 s]
/proc/loadavg: 10.27 9.77 7.69 7/498 17454
/proc/meminfo: memFree=24795884/32951124 swapFree=53501244/67111528
[pid=17454] ppid=17452 vsize=135748 CPUtime=0.18 cores=1,3,5,7
/proc/17454/stat : 17454 (python) S 17452 17454 16982 0 -1 4202496 5962 934 1 0 14 4 0 0 18 0 1 0 657986928 139005952 4150 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634511365 0 2147483391 16781312 2 18446744071699199166 0 0 17 3 0 0 0
/proc/17454/statm: 33937 4150 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 135748

[startup+0.70014 s]
/proc/loadavg: 10.27 9.77 7.69 7/498 17454
/proc/meminfo: memFree=24795884/32951124 swapFree=53501244/67111528
[pid=17454] ppid=17452 vsize=284344 CPUtime=0.45 cores=1,3,5,7
/proc/17454/stat : 17454 (python) R 17452 17454 16982 0 -1 4202496 10704 934 1 0 36 9 0 0 17 0 1 0 657986928 291168256 8388 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634176197 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/17454/statm: 71086 8388 1761 364 0 7010 0
Current children cumulated CPU time (s) 0.45
Current children cumulated vsize (KiB) 284344

[startup+1.501 s]
/proc/loadavg: 10.27 9.77 7.69 8/499 17459
/proc/meminfo: memFree=24747392/32951124 swapFree=53501244/67111528
[pid=17454] ppid=17452 vsize=302956 CPUtime=1.25 cores=1,3,5,7
/proc/17454/stat : 17454 (python) R 17452 17454 16982 0 -1 4202496 18307 934 1 0 115 10 0 0 17 0 1 0 657986928 310226944 13124 33554432000 4194304 5683644 140736153411008 18446744073709551615 46944329419054 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/17454/statm: 75739 13124 1826 364 0 11663 0
Current children cumulated CPU time (s) 1.25
Current children cumulated vsize (KiB) 302956

[startup+3.10072 s]
/proc/loadavg: 10.27 9.77 7.69 8/499 17459
/proc/meminfo: memFree=24747260/32951124 swapFree=53501244/67111528
[pid=17454] ppid=17452 vsize=302956 CPUtime=2.85 cores=1,3,5,7
/proc/17454/stat : 17454 (python) R 17452 17454 16982 0 -1 4202496 18308 934 1 0 275 10 0 0 20 0 1 0 657986928 310226944 13125 33554432000 4194304 5683644 140736153411008 18446744073709551615 4562986 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/17454/statm: 75739 13125 1827 364 0 11663 0
Current children cumulated CPU time (s) 2.85
Current children cumulated vsize (KiB) 302956

[startup+6.30113 s]
/proc/loadavg: 10.01 9.72 7.68 11/511 17472
/proc/meminfo: memFree=24693348/32951124 swapFree=53501244/67111528
[pid=17454] ppid=17452 vsize=342040 CPUtime=4.94 cores=1,3,5,7
/proc/17454/stat : 17454 (python) S 17452 17454 16982 0 -1 4202496 21121 934 1 0 482 12 0 0 25 0 5 0 657986928 350248960 12699 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646342091 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/17454/statm: 85510 12699 1873 364 0 21419 0
[pid=17461] ppid=17454 vsize=301028 CPUtime=0 cores=1,3,5,7
/proc/17461/stat : 17461 (python) S 17454 17454 16982 0 -1 4202560 739 0 0 0 0 0 0 0 17 0 1 0 657987443 308252672 11163 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634535439 0 0 16781312 514 15454088931477258504 0 0 17 7 0 0 0
/proc/17461/statm: 75257 11163 353 364 0 11175 0
[pid=17463] ppid=17461 vsize=268424 CPUtime=0.62 cores=1,3,5,7
/proc/17463/stat : 17463 (hybridGM3) R 17461 17463 17463 0 -1 4202496 7023 0 0 0 62 0 0 0 25 0 1 0 657987444 274866176 6758 33554432000 134512640 135204385 4287655440 18446744073709551615 134537383 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/17463/statm: 67106 6758 64 169 0 66936 0
[pid=17464] ppid=17454 vsize=311284 CPUtime=0 cores=1,3,5,7
/proc/17464/stat : 17464 (python) S 17454 17454 16982 0 -1 4202560 736 0 0 0 0 0 0 0 20 0 1 0 657987444 318754816 11161 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634535494 0 0 16781312 514 386980818272124927 0 0 17 7 0 0 0
/proc/17464/statm: 77821 11161 347 364 0 13736 0
[pid=17466] ppid=17464 vsize=268424 CPUtime=1.1 cores=1,3,5,7
/proc/17466/stat : 17466 (hybridGM3) R 17464 17466 17466 0 -1 4202496 7130 0 0 0 109 1 0 0 25 0 1 0 657987445 274866176 6868 33554432000 134512640 135204385 4288356784 18446744073709551615 134546082 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/17466/statm: 67106 6868 64 169 0 66936 0
[pid=17467] ppid=17454 vsize=321540 CPUtime=0 cores=1,3,5,7
/proc/17467/stat : 17467 (python) S 17454 17454 16982 0 -1 4202560 715 0 0 0 0 0 0 0 25 0 1 0 657987445 329256960 11161 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634535494 0 0 16781312 514 5641844305308942335 0 0 17 3 0 0 0
/proc/17467/statm: 80385 11161 343 364 0 16297 0
[pid=17471] ppid=17467 vsize=268416 CPUtime=0.49 cores=1,3,5,7
/proc/17471/stat : 17471 (hybridGM3) R 17467 17471 17471 0 -1 4202496 3662 0 0 0 49 0 0 0 25 0 1 0 657987447 274857984 3402 33554432000 134512640 135204385 4287440112 18446744073709551615 134543975 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/17471/statm: 67104 3402 57 169 0 66934 0
[pid=17469] ppid=17454 vsize=331796 CPUtime=0 cores=1,3,5,7
/proc/17469/stat : 17469 (python) S 17454 17454 16982 0 -1 4202560 751 0 0 0 0 0 0 0 20 0 1 0 657987447 339759104 11169 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634535494 0 0 16781312 514 7573840991572983807 0 0 17 3 0 0 0
/proc/17469/statm: 82949 11169 347 364 0 18858 0
[pid=17472] ppid=17469 vsize=268428 CPUtime=1.1 cores=1,3,5,7
/proc/17472/stat : 17472 (hybridGM3) R 17469 17472 17472 0 -1 4202496 7131 0 0 0 109 1 0 0 25 0 1 0 657987447 274870272 6871 33554432000 134512640 135204385 4286682608 18446744073709551615 134536605 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/17472/statm: 67107 6871 64 169 0 66937 0
[pid=17454/tid=17462] ppid=17452 vsize=342040 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17462/stat : 17462 (python) S 17452 17454 16982 0 -1 4202560 26 934 0 0 0 0 0 0 25 0 5 0 657987444 350248960 12699 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 7 0 0 0
[pid=17454/tid=17465] ppid=17452 vsize=342040 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17465/stat : 17465 (python) S 17452 17454 16982 0 -1 4202560 25 934 0 0 0 0 0 0 25 0 5 0 657987445 350248960 12699 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 7 0 0 0
[pid=17454/tid=17468] ppid=17452 vsize=342040 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17468/stat : 17468 (python) S 17452 17454 16982 0 -1 4202560 40 934 0 0 0 0 0 0 25 0 5 0 657987446 350248960 12699 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17470] ppid=17452 vsize=342040 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17470/stat : 17470 (python) S 17452 17454 16982 0 -1 4202560 20 934 0 0 0 0 0 0 25 0 5 0 657987447 350248960 12699 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 8.25
Current children cumulated vsize (KiB) 2681380

[startup+12.701 s]
/proc/loadavg: 10.01 9.73 7.70 11/512 17473
/proc/meminfo: memFree=24620020/32951124 swapFree=53501244/67111528
[pid=17454] ppid=17452 vsize=342040 CPUtime=4.94 cores=1,3,5,7
/proc/17454/stat : 17454 (python) S 17452 17454 16982 0 -1 4202496 21121 934 1 0 482 12 0 0 25 0 5 0 657986928 350248960 12699 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646342091 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/17454/statm: 85510 12699 1873 364 0 21419 0
[pid=17461] ppid=17454 vsize=301028 CPUtime=0.02 cores=1,3,5,7
/proc/17461/stat : 17461 (python) S 17454 17454 16982 0 -1 4202560 906 0 0 0 1 1 0 0 15 0 1 0 657987443 308252672 11186 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634535439 0 0 16781312 514 4082649405948133640 0 0 17 7 0 0 0
/proc/17461/statm: 75257 11186 376 364 0 11175 0
[pid=17463] ppid=17461 vsize=268424 CPUtime=3.83 cores=1,3,5,7
/proc/17463/stat : 17463 (hybridGM3) R 17461 17463 17463 0 -1 4202496 7136 0 0 0 382 1 0 0 25 0 1 0 657987444 274866176 6871 33554432000 134512640 135204385 4287655440 18446744073709551615 134545975 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/17463/statm: 67106 6871 64 169 0 66936 0
[pid=17464] ppid=17454 vsize=311284 CPUtime=0.02 cores=1,3,5,7
/proc/17464/stat : 17464 (python) S 17454 17454 16982 0 -1 4202560 910 0 0 0 1 1 0 0 18 0 1 0 657987444 318754816 11183 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634535494 0 0 16781312 514 386980818272092421 0 0 17 7 0 0 0
/proc/17464/statm: 77821 11183 369 364 0 13736 0
[pid=17466] ppid=17464 vsize=268424 CPUtime=7.35 cores=1,3,5,7
/proc/17466/stat : 17466 (hybridGM3) R 17464 17466 17466 0 -1 4202496 7134 0 0 0 734 1 0 0 25 0 1 0 657987445 274866176 6872 33554432000 134512640 135204385 4288356784 18446744073709551615 134537483 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/17466/statm: 67106 6872 64 169 0 66936 0
[pid=17467] ppid=17454 vsize=321540 CPUtime=0.01 cores=1,3,5,7
/proc/17467/stat : 17467 (python) S 17454 17454 16982 0 -1 4202560 902 0 0 0 0 1 0 0 15 0 1 0 657987445 329256960 11188 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634535494 0 0 16781312 514 5641844305308909831 0 0 17 3 0 0 0
/proc/17467/statm: 80385 11188 370 364 0 16297 0
[pid=17471] ppid=17467 vsize=268420 CPUtime=3.69 cores=1,3,5,7
/proc/17471/stat : 17471 (hybridGM3) R 17467 17471 17471 0 -1 4202496 7129 0 0 0 368 1 0 0 25 0 1 0 657987447 274862080 6869 33554432000 134512640 135204385 4287440112 18446744073709551615 134545866 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/17471/statm: 67105 6869 64 169 0 66935 0
[pid=17469] ppid=17454 vsize=331796 CPUtime=0.02 cores=1,3,5,7
/proc/17469/stat : 17469 (python) S 17454 17454 16982 0 -1 4202560 916 0 0 0 1 1 0 0 18 0 1 0 657987447 339759104 11192 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634535494 0 0 16781312 514 7573840991572951302 0 0 17 3 0 0 0
/proc/17469/statm: 82949 11192 370 364 0 18858 0
[pid=17472] ppid=17469 vsize=268428 CPUtime=7.46 cores=1,3,5,7
/proc/17472/stat : 17472 (hybridGM3) R 17469 17472 17472 0 -1 4202496 7131 0 0 0 745 1 0 0 25 0 1 0 657987447 274870272 6871 33554432000 134512640 135204385 4286682608 18446744073709551615 134536656 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/17472/statm: 67107 6871 64 169 0 66937 0
[pid=17454/tid=17462] ppid=17452 vsize=342040 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17462/stat : 17462 (python) S 17452 17454 16982 0 -1 4202560 26 934 0 0 0 0 0 0 25 0 5 0 657987444 350248960 12699 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 7 0 0 0
[pid=17454/tid=17465] ppid=17452 vsize=342040 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17465/stat : 17465 (python) S 17452 17454 16982 0 -1 4202560 25 934 0 0 0 0 0 0 25 0 5 0 657987445 350248960 12699 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 7 0 0 0
[pid=17454/tid=17468] ppid=17452 vsize=342040 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17468/stat : 17468 (python) S 17452 17454 16982 0 -1 4202560 40 934 0 0 0 0 0 0 25 0 5 0 657987446 350248960 12699 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17470] ppid=17452 vsize=342040 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17470/stat : 17470 (python) S 17452 17454 16982 0 -1 4202560 20 934 0 0 0 0 0 0 25 0 5 0 657987447 350248960 12699 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 27.34
Current children cumulated vsize (KiB) 2681384

[startup+25.5007 s]
/proc/loadavg: 10.00 9.74 7.73 11/512 17473
/proc/meminfo: memFree=24619372/32951124 swapFree=53501244/67111528
[pid=17454] ppid=17452 vsize=342040 CPUtime=4.94 cores=1,3,5,7
/proc/17454/stat : 17454 (python) S 17452 17454 16982 0 -1 4202496 21121 934 1 0 482 12 0 0 25 0 5 0 657986928 350248960 12699 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646342091 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/17454/statm: 85510 12699 1873 364 0 21419 0
[pid=17461] ppid=17454 vsize=301028 CPUtime=0.05 cores=1,3,5,7
/proc/17461/stat : 17461 (python) S 17454 17454 16982 0 -1 4202560 912 0 0 0 3 2 0 0 15 0 1 0 657987443 308252672 11186 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634535439 0 0 16781312 514 1215672832227246344 0 0 17 7 0 0 0
/proc/17461/statm: 75257 11186 376 364 0 11175 0
[pid=17463] ppid=17461 vsize=268424 CPUtime=10.2 cores=1,3,5,7
/proc/17463/stat : 17463 (hybridGM3) R 17461 17463 17463 0 -1 4202496 7136 0 0 0 1019 1 0 0 25 0 1 0 657987444 274866176 6871 33554432000 134512640 135204385 4287655440 18446744073709551615 134537483 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/17463/statm: 67106 6871 64 169 0 66936 0
[pid=17464] ppid=17454 vsize=311284 CPUtime=0.06 cores=1,3,5,7
/proc/17464/stat : 17464 (python) S 17454 17454 16982 0 -1 4202560 917 0 0 0 3 3 0 0 18 0 1 0 657987444 318754816 11184 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634535494 0 0 16781312 514 1215672832227246341 0 0 17 7 0 0 0
/proc/17464/statm: 77821 11184 370 364 0 13736 0
[pid=17466] ppid=17464 vsize=268424 CPUtime=19.87 cores=1,3,5,7
/proc/17466/stat : 17466 (hybridGM3) R 17464 17466 17466 0 -1 4202496 7134 0 0 0 1986 1 0 0 25 0 1 0 657987445 274866176 6872 33554432000 134512640 135204385 4288356784 18446744073709551615 134537383 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/17466/statm: 67106 6872 64 169 0 66936 0
[pid=17467] ppid=17454 vsize=321540 CPUtime=0.06 cores=1,3,5,7
/proc/17467/stat : 17467 (python) S 17454 17454 16982 0 -1 4202560 908 0 0 0 3 3 0 0 15 0 1 0 657987445 329256960 11188 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634535494 0 0 16781312 514 1215672832227246343 0 0 17 3 0 0 0
/proc/17467/statm: 80385 11188 370 364 0 16297 0
[pid=17471] ppid=17467 vsize=268420 CPUtime=12.03 cores=1,3,5,7
/proc/17471/stat : 17471 (hybridGM3) R 17467 17471 17471 0 -1 4202496 7129 0 0 0 1202 1 0 0 25 0 1 0 657987447 274862080 6869 33554432000 134512640 135204385 4287440112 18446744073709551615 134537272 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/17471/statm: 67105 6869 64 169 0 66935 0
[pid=17469] ppid=17454 vsize=331796 CPUtime=0.06 cores=1,3,5,7
/proc/17469/stat : 17469 (python) S 17454 17454 16982 0 -1 4202560 922 0 0 0 3 3 0 0 18 0 1 0 657987447 339759104 11192 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634535494 0 0 16781312 514 1215672832227246342 0 0 17 3 0 0 0
/proc/17469/statm: 82949 11192 370 364 0 18858 0
[pid=17472] ppid=17469 vsize=268428 CPUtime=18.28 cores=1,3,5,7
/proc/17472/stat : 17472 (hybridGM3) R 17469 17472 17472 0 -1 4202496 7131 0 0 0 1827 1 0 0 25 0 1 0 657987447 274870272 6871 33554432000 134512640 135204385 4286682608 18446744073709551615 134536737 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/17472/statm: 67107 6871 64 169 0 66937 0
[pid=17454/tid=17462] ppid=17452 vsize=342040 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17462/stat : 17462 (python) S 17452 17454 16982 0 -1 4202560 26 934 0 0 0 0 0 0 25 0 5 0 657987444 350248960 12699 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 7 0 0 0
[pid=17454/tid=17465] ppid=17452 vsize=342040 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17465/stat : 17465 (python) S 17452 17454 16982 0 -1 4202560 25 934 0 0 0 0 0 0 25 0 5 0 657987445 350248960 12699 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 7 0 0 0
[pid=17454/tid=17468] ppid=17452 vsize=342040 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17468/stat : 17468 (python) S 17452 17454 16982 0 -1 4202560 40 934 0 0 0 0 0 0 25 0 5 0 657987446 350248960 12699 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0

################
# More data... #
################

/proc/17466/stat : 17466 (hybridGM3) T 17464 17466 17466 0 -1 4202496 7137 0 0 0 29129 1 0 0 25 0 1 0 657987445 274866176 6875 33554432000 134512640 135204385 4288356784 18446744073709551615 134545851 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/17466/statm: 67106 6875 64 169 0 66936 0
[pid=17464/tid=17486] ppid=17454 vsize=311284 CPUtime=0 cores=1,3,5,7
/proc/17464/task/17486/stat : 17486 (python) S 17454 17454 16982 0 -1 4202560 46 0 0 0 0 0 0 0 18 0 2 0 658001561 318754816 11214 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 514 18446604465887696128 0 0 -1 7 0 0 0
[pid=17467] ppid=17454 vsize=321540 CPUtime=1.53 cores=1,3,5,7
/proc/17467/stat : 17467 (python) S 17454 17454 16982 0 -1 4202560 1141 0 0 0 75 78 0 0 15 0 2 0 657987445 329256960 11218 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/17467/statm: 80385 11218 399 364 0 16297 0
[pid=17471] ppid=17467 vsize=268420 CPUtime=291.85 cores=1,3,5,7
/proc/17471/stat : 17471 (hybridGM3) T 17467 17471 17471 0 -1 4202496 7132 0 0 0 29184 1 0 0 25 0 1 0 657987447 274862080 6872 33554432000 134512640 135204385 4287440112 18446744073709551615 134536663 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/17471/statm: 67105 6872 64 169 0 66935 0
[pid=17467/tid=17488] ppid=17454 vsize=321540 CPUtime=0 cores=1,3,5,7
/proc/17467/task/17488/stat : 17488 (python) S 17454 17454 16982 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 658002336 329256960 11218 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 514 18446604467111189376 0 0 -1 3 0 0 0
[pid=17469] ppid=17454 vsize=331796 CPUtime=1.51 cores=1,3,5,7
/proc/17469/stat : 17469 (python) S 17454 17454 16982 0 -1 4202560 1168 0 0 0 75 76 0 0 18 0 2 0 657987447 339759104 11222 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/17469/statm: 82949 11222 399 364 0 18858 0
[pid=17472] ppid=17469 vsize=268428 CPUtime=291.53 cores=1,3,5,7
/proc/17472/stat : 17472 (hybridGM3) T 17469 17472 17472 0 -1 4202496 7136 0 0 0 29152 1 0 0 25 0 1 0 657987447 274870272 6876 33554432000 134512640 135204385 4286682608 18446744073709551615 134536993 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/17472/statm: 67107 6876 64 169 0 66937 0
[pid=17469/tid=17589] ppid=17454 vsize=331796 CPUtime=0 cores=1,3,5,7
/proc/17469/task/17589/stat : 17589 (python) S 17454 17454 16982 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 658014399 339759104 11222 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 514 18446604454676115520 0 0 -1 3 0 0 0
[pid=17623] ppid=17454 vsize=342052 CPUtime=1.5 cores=1,3,5,7
/proc/17623/stat : 17623 (python) S 17454 17454 16982 0 -1 4202560 1159 0 0 0 75 75 0 0 18 0 2 0 658020771 350261248 11226 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634535494 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/17623/statm: 85513 11226 399 364 0 21419 0
[pid=17625] ppid=17623 vsize=460952 CPUtime=379.26 cores=1,3,5,7
/proc/17625/stat : 17625 (TNM) R 17623 17625 17625 0 -1 4202496 4393 0 0 0 37925 1 0 0 25 0 1 0 658020771 472014848 4133 33554432000 134512640 135008748 4289921456 18446744073709551615 134524674 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/17625/statm: 115238 4133 46 122 0 114138 0
[pid=17623/tid=17683] ppid=17454 vsize=342052 CPUtime=0 cores=1,3,5,7
/proc/17623/task/17683/stat : 17683 (python) S 17454 17454 16982 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 658031407 350261248 11226 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 514 18446604467339634816 0 0 -1 3 0 0 0
[pid=17640] ppid=17454 vsize=352308 CPUtime=1.92 cores=1,3,5,7
/proc/17640/stat : 17640 (python) S 17454 17454 16982 0 -1 4202560 1170 0 0 0 98 94 0 0 15 0 2 0 658026253 360763392 11230 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/17640/statm: 88077 11230 399 364 0 23980 0
[pid=17642] ppid=17640 vsize=268420 CPUtime=389.17 cores=1,3,5,7
/proc/17642/stat : 17642 (hybridGM3) T 17640 17642 17642 0 -1 4202496 7135 0 0 0 38915 2 0 0 25 0 1 0 658026263 274862080 6874 33554432000 134512640 135204385 4286820304 18446744073709551615 134537272 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/17642/statm: 67105 6874 64 169 0 66935 0
[pid=17640/tid=17804] ppid=17454 vsize=352308 CPUtime=0 cores=1,3,5,7
/proc/17640/task/17804/stat : 17804 (python) S 17454 17454 16982 0 -1 4202560 50 0 0 0 0 0 0 0 15 0 2 0 658052573 360763392 11230 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 514 18446604463087254336 0 0 -1 3 0 0 0
[pid=17644] ppid=17454 vsize=362564 CPUtime=1.22 cores=1,3,5,7
/proc/17644/stat : 17644 (python) S 17454 17454 16982 0 -1 4202560 1167 0 0 0 58 64 0 0 15 0 2 0 658029952 371265536 11234 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/17644/statm: 90641 11234 399 364 0 26541 0
[pid=17646] ppid=17644 vsize=268424 CPUtime=293.61 cores=1,3,5,7
/proc/17646/stat : 17646 (hybridGM3) T 17644 17646 17646 0 -1 4202496 7134 0 0 0 29360 1 0 0 25 0 1 0 658029953 274866176 6874 33554432000 134512640 135204385 4287419088 18446744073709551615 134547418 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/17646/statm: 67106 6874 64 169 0 66936 0
[pid=17644/tid=17790] ppid=17454 vsize=362564 CPUtime=0 cores=1,3,5,7
/proc/17644/task/17790/stat : 17790 (python) S 17454 17454 16982 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 658043030 371265536 11234 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 514 18446604464548930496 0 0 -1 3 0 0 0
[pid=17648] ppid=17454 vsize=372820 CPUtime=1.44 cores=1,3,5,7
/proc/17648/stat : 17648 (python) S 17454 17454 16982 0 -1 4202560 1205 0 0 0 71 73 0 0 15 0 2 0 658031035 381767680 11244 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/17648/statm: 93205 11244 405 364 0 29102 0
[pid=17650] ppid=17648 vsize=268420 CPUtime=292.13 cores=1,3,5,7
/proc/17650/stat : 17650 (hybridGM3) T 17648 17650 17650 0 -1 4202496 7138 0 0 0 29212 1 0 0 25 0 1 0 658031035 274862080 6871 33554432000 134512640 135204385 4293622784 18446744073709551615 134537440 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/17650/statm: 67105 6871 64 169 0 66935 0
[pid=17648/tid=17813] ppid=17454 vsize=372820 CPUtime=0 cores=1,3,5,7
/proc/17648/task/17813/stat : 17813 (python) S 17454 17454 16982 0 -1 4202560 49 0 0 0 0 0 0 0 15 0 2 0 658059154 381767680 11244 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 514 18446604469835459584 0 0 -1 3 0 0 0
[pid=17684] ppid=17454 vsize=383076 CPUtime=1.86 cores=1,3,5,7
/proc/17684/stat : 17684 (python) S 17454 17454 16982 0 -1 4202560 1199 0 0 0 92 94 0 0 16 0 2 0 658031408 392269824 11248 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/17684/statm: 95769 11248 405 364 0 31663 0
[pid=17686] ppid=17684 vsize=268424 CPUtime=389.56 cores=1,3,5,7
/proc/17686/stat : 17686 (hybridGM3) T 17684 17686 17686 0 -1 4202496 7144 0 0 0 38955 1 0 0 25 0 1 0 658031409 274866176 6877 33554432000 134512640 135204385 4289616320 18446744073709551615 134546082 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/17686/statm: 67106 6877 64 169 0 66936 0
[pid=17684/tid=17811] ppid=17454 vsize=383076 CPUtime=0 cores=1,3,5,7
/proc/17684/task/17811/stat : 17811 (python) S 17454 17454 16982 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 658057129 392269824 11248 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 514 18446604469835457792 0 0 -1 3 0 0 0
[pid=17805] ppid=17454 vsize=393332 CPUtime=1.07 cores=1,3,5,7
/proc/17805/stat : 17805 (python) S 17454 17454 16982 0 -1 4202560 1170 0 0 0 56 51 0 0 16 0 2 0 658053168 402771968 11246 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/17805/statm: 98333 11246 399 364 0 34224 0
[pid=17807] ppid=17805 vsize=268428 CPUtime=195.84 cores=1,3,5,7
/proc/17807/stat : 17807 (hybridGM3) T 17805 17807 17807 0 -1 4202496 7140 0 0 0 19583 1 0 0 25 0 1 0 658053168 274870272 6875 33554432000 134512640 135204385 4286937408 18446744073709551615 134537412 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/17807/statm: 67107 6875 64 169 0 66937 0
[pid=17805/tid=17837] ppid=17454 vsize=393332 CPUtime=0 cores=1,3,5,7
/proc/17805/task/17837/stat : 17837 (python) S 17454 17454 16982 0 -1 4202560 47 0 0 0 0 0 0 0 16 0 2 0 658082496 402771968 11246 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 514 18446604460603419456 0 0 -1 3 0 0 0
[pid=17821] ppid=17454 vsize=403588 CPUtime=0.56 cores=1,3,5,7
/proc/17821/stat : 17821 (python) S 17454 17454 16982 0 -1 4202560 1151 0 0 0 28 28 0 0 16 0 2 0 658070496 413274112 11250 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/17821/statm: 100897 11250 399 364 0 36785 0
[pid=17823] ppid=17821 vsize=1025020 CPUtime=99.89 cores=1,3,5,7
/proc/17823/stat : 17823 (gnovelty+2) T 17821 17823 17823 0 -1 4202496 2612 0 0 0 9989 0 0 0 25 0 1 0 658070496 1049620480 2314 33554432000 134512640 135663704 4294555072 18446744073709551615 134525187 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/17823/statm: 256255 2314 113 282 0 255971 0
[pid=17821/tid=17842] ppid=17454 vsize=403588 CPUtime=0 cores=1,3,5,7
/proc/17821/task/17842/stat : 17842 (python) S 17454 17454 16982 0 -1 4202560 48 0 0 0 0 0 0 0 16 0 2 0 658085971 413274112 11250 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 514 18446604459366495552 0 0 -1 3 0 0 0
[pid=17843] ppid=17454 vsize=413844 CPUtime=0.54 cores=1,3,5,7
/proc/17843/stat : 17843 (python) S 17454 17454 16982 0 -1 4202560 990 0 0 0 26 28 0 0 18 0 1 0 658085972 423776256 11224 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634535494 0 0 16781312 514 1215672832227246341 0 0 17 3 0 0 0
/proc/17843/statm: 103461 11224 370 364 0 39346 0
[pid=17845] ppid=17843 vsize=460952 CPUtime=99.52 cores=1,3,5,7
/proc/17845/stat : 17845 (TNM) R 17843 17845 17845 0 -1 4202496 4405 0 0 0 9952 0 0 0 25 0 1 0 658085973 472014848 4134 33554432000 134512640 135008748 4288396032 18446744073709551615 134529448 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/17845/statm: 115238 4134 46 122 0 114138 0
[pid=17847] ppid=17454 vsize=424100 CPUtime=0.45 cores=1,3,5,7
/proc/17847/stat : 17847 (python) S 17454 17454 16982 0 -1 4202560 995 0 0 0 24 21 0 0 18 0 1 0 658088605 434278400 11228 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634535494 0 0 16781312 514 1215672832227246341 0 0 17 3 0 0 0
/proc/17847/statm: 106025 11228 370 364 0 41907 0
[pid=17849] ppid=17847 vsize=460952 CPUtime=95.58 cores=1,3,5,7
/proc/17849/stat : 17849 (TNM) R 17847 17849 17849 0 -1 4202496 4402 0 0 0 9558 0 0 0 25 0 1 0 658088605 472014848 4134 33554432000 134512640 135008748 4292693600 18446744073709551615 134529422 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/17849/statm: 115238 4134 46 122 0 114138 0
[pid=17851] ppid=17454 vsize=434356 CPUtime=0.24 cores=1,3,5,7
/proc/17851/stat : 17851 (python) S 17454 17454 16982 0 -1 4202560 991 0 0 0 12 12 0 0 15 0 1 0 658094040 444780544 11232 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634535494 0 0 16781312 514 1215672832227246341 0 0 17 3 0 0 0
/proc/17851/statm: 108589 11232 370 364 0 44468 0
[pid=17853] ppid=17851 vsize=268428 CPUtime=57.83 cores=1,3,5,7
/proc/17853/stat : 17853 (hybridGM3) R 17851 17853 17853 0 -1 4202496 7137 0 0 0 5782 1 0 0 25 0 1 0 658094041 274870272 6869 33554432000 134512640 135204385 4288329504 18446744073709551615 134537249 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/17853/statm: 67107 6869 64 169 0 66937 0
[pid=17454/tid=17462] ppid=17452 vsize=444600 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17462/stat : 17462 (python) S 17452 17454 16982 0 -1 4202560 32 934 0 0 0 0 0 0 18 0 15 0 657987444 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 7 0 0 0
[pid=17454/tid=17465] ppid=17452 vsize=444600 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17465/stat : 17465 (python) S 17452 17454 16982 0 -1 4202560 29 934 0 0 0 0 0 0 18 0 15 0 657987445 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 7 0 0 0
[pid=17454/tid=17468] ppid=17452 vsize=444600 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17468/stat : 17468 (python) S 17452 17454 16982 0 -1 4202560 41 934 0 0 0 0 0 0 18 0 15 0 657987446 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17470] ppid=17452 vsize=444600 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17470/stat : 17470 (python) S 17452 17454 16982 0 -1 4202560 20 934 0 0 0 0 0 0 18 0 15 0 657987447 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17624] ppid=17452 vsize=444600 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17624/stat : 17624 (python) S 17452 17454 16982 0 -1 4202560 32 934 0 0 0 0 0 0 18 0 15 0 658020771 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17641] ppid=17452 vsize=444600 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17641/stat : 17641 (python) S 17452 17454 16982 0 -1 4202560 36 934 0 0 0 0 0 0 18 0 15 0 658026253 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17645] ppid=17452 vsize=444600 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17645/stat : 17645 (python) S 17452 17454 16982 0 -1 4202560 36 934 0 0 0 0 0 0 18 0 15 0 658029953 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17649] ppid=17452 vsize=444600 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17649/stat : 17649 (python) S 17452 17454 16982 0 -1 4202560 30 934 0 0 0 0 0 0 18 0 15 0 658031035 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17685] ppid=17452 vsize=444600 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17685/stat : 17685 (python) S 17452 17454 16982 0 -1 4202560 33 934 0 0 0 0 0 0 18 0 15 0 658031409 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17806] ppid=17452 vsize=444600 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17806/stat : 17806 (python) S 17452 17454 16982 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 15 0 658053168 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17822] ppid=17452 vsize=444600 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17822/stat : 17822 (python) S 17452 17454 16982 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 15 0 658070496 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17844] ppid=17452 vsize=444600 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17844/stat : 17844 (python) S 17452 17454 16982 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 15 0 658085972 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17848] ppid=17452 vsize=444600 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17848/stat : 17848 (python) S 17452 17454 16982 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 15 0 658088605 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17852] ppid=17452 vsize=444600 CPUtime=0 cores=1,3,5,7
/proc/17454/task/17852/stat : 17852 (python) S 17452 17454 16982 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 15 0 658094041 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3482.66
Current children cumulated vsize (KiB) 10694648

[startup+1139.1 s]
/proc/loadavg: 11.16 11.16 10.07 13/544 17860
/proc/meminfo: memFree=24171092/32951124 swapFree=53501244/67111528
[pid=17454] ppid=17452 vsize=444600 CPUtime=3329.55 cores=1,3,5,7
/proc/17454/stat : 17454 (python) S 17452 17454 16982 0 -1 4202496 30997 91023 1 0 503 15 331617 820 17 0 15 0 657986928 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646345535 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17454/statm: 111150 12760 1894 364 0 47029 0
[pid=17847] ppid=17454 vsize=424100 CPUtime=0.45 cores=1,3,5,7
/proc/17847/stat : 17847 (python) S 17454 17454 16982 0 -1 4202560 995 0 0 0 24 21 0 0 18 0 1 0 658088605 434278400 11228 33554432000 4194304 5683644 140736153411008 18446744073709551615 216634535494 0 0 16781312 514 1215672832227246341 0 0 17 3 0 0 0
/proc/17847/statm: 106025 11228 370 364 0 41907 0
[pid=17849] ppid=17847 vsize=460952 CPUtime=95.8 cores=1,3,5,7
/proc/17849/stat : 17849 (TNM) R 17847 17849 17849 0 -1 4202496 4402 0 0 0 9580 0 0 0 25 0 1 0 658088605 472014848 4134 33554432000 134512640 135008748 4292693600 18446744073709551615 134527795 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/17849/statm: 115238 4134 46 122 0 114138 0
[pid=17851] ppid=17454 vsize=434356 CPUtime=58.47 cores=1,3,5,7
/proc/17851/stat : 17851 (python) R 17454 17454 16982 0 -1 4202560 1031 7137 0 0 13 12 5821 1 15 0 1 0 658094040 444780544 11237 33554432000 4194304 5683644 140736153411008 18446744073709551615 4539735 0 0 16781824 2 0 0 0 17 5 0 0 0
/proc/17851/statm: 108589 11242 380 364 0 44468 0
[pid=17454/tid=17462] ppid=17452 vsize=444600 CPUtime=3324.37 cores=1,3,5,7
/proc/17454/task/17462/stat : 17462 (python) S 17452 17454 16982 0 -1 4202560 32 91023 0 0 0 0 331617 820 18 0 15 0 657987444 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 7 0 0 0
[pid=17454/tid=17465] ppid=17452 vsize=444600 CPUtime=3324.37 cores=1,3,5,7
/proc/17454/task/17465/stat : 17465 (python) S 17452 17454 16982 0 -1 4202560 29 91023 0 0 0 0 331617 820 18 0 15 0 657987445 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 7 0 0 0
[pid=17454/tid=17468] ppid=17452 vsize=444600 CPUtime=3324.37 cores=1,3,5,7
/proc/17454/task/17468/stat : 17468 (python) S 17452 17454 16982 0 -1 4202560 41 91023 0 0 0 0 331617 820 18 0 15 0 657987446 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17470] ppid=17452 vsize=444600 CPUtime=3324.37 cores=1,3,5,7
/proc/17454/task/17470/stat : 17470 (python) S 17452 17454 16982 0 -1 4202560 20 91023 0 0 0 0 331617 820 18 0 15 0 657987447 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17624] ppid=17452 vsize=444600 CPUtime=3324.37 cores=1,3,5,7
/proc/17454/task/17624/stat : 17624 (python) S 17452 17454 16982 0 -1 4202560 32 91023 0 0 0 0 331617 820 18 0 15 0 658020771 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17641] ppid=17452 vsize=444600 CPUtime=3324.37 cores=1,3,5,7
/proc/17454/task/17641/stat : 17641 (python) S 17452 17454 16982 0 -1 4202560 36 91023 0 0 0 0 331617 820 18 0 15 0 658026253 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17645] ppid=17452 vsize=444600 CPUtime=3324.37 cores=1,3,5,7
/proc/17454/task/17645/stat : 17645 (python) S 17452 17454 16982 0 -1 4202560 36 91023 0 0 0 0 331617 820 18 0 15 0 658029953 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17649] ppid=17452 vsize=444600 CPUtime=3324.37 cores=1,3,5,7
/proc/17454/task/17649/stat : 17649 (python) S 17452 17454 16982 0 -1 4202560 30 91023 0 0 0 0 331617 820 18 0 15 0 658031035 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17685] ppid=17452 vsize=444600 CPUtime=3324.37 cores=1,3,5,7
/proc/17454/task/17685/stat : 17685 (python) S 17452 17454 16982 0 -1 4202560 33 91023 0 0 0 0 331617 820 18 0 15 0 658031409 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17806] ppid=17452 vsize=444600 CPUtime=3324.37 cores=1,3,5,7
/proc/17454/task/17806/stat : 17806 (python) S 17452 17454 16982 0 -1 4202560 28 91023 0 0 0 0 331617 820 20 0 15 0 658053168 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17822] ppid=17452 vsize=444600 CPUtime=3324.37 cores=1,3,5,7
/proc/17454/task/17822/stat : 17822 (python) S 17452 17454 16982 0 -1 4202560 30 91023 0 0 0 0 331617 820 20 0 15 0 658070496 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17844] ppid=17452 vsize=444600 CPUtime=3324.37 cores=1,3,5,7
/proc/17454/task/17844/stat : 17844 (python) S 17452 17454 16982 0 -1 4202560 28 91023 0 0 0 0 331617 820 20 0 15 0 658085972 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17848] ppid=17452 vsize=444600 CPUtime=3324.37 cores=1,3,5,7
/proc/17454/task/17848/stat : 17848 (python) S 17452 17454 16982 0 -1 4202560 28 91023 0 0 0 0 331617 820 20 0 15 0 658088605 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
[pid=17454/tid=17852] ppid=17452 vsize=444600 CPUtime=3324.37 cores=1,3,5,7
/proc/17454/task/17852/stat : 17852 (python) S 17452 17454 16982 0 -1 4202560 29 91023 0 0 0 0 331617 820 20 0 15 0 658094041 455270400 12760 33554432000 4194304 5683644 140736153411008 18446744073709551615 216646338993 0 0 16781312 2 18446604467019464896 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3484.27
Current children cumulated vsize (KiB) 1764008

Child status: 0
Real time (s): 1139.13
CPU time (s): 3484.35
CPU user time (s): 3475.62
CPU system time (s): 8.73067
CPU usage (%): 305.878
Max. virtual memory (cumulated for all children) (KiB): 10694652

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3475.62
system time used= 8.73067
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 142292
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 9097
involuntary context switches= 43797

runsolver used 7.10692 second user time and 17.8293 second system time

The end

Launcher Data 

Begin job on node103 at 2011-05-15 14:50:43
IDJOB=3378219
IDBENCH=82628
IDSOLVER=1859
FILE ID=node103/3378219-1305463843
RUNJOBID= node103-1305462668-17001
PBS_JOBID= 13324971
Free space on /tmp= 72936 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v25000-c105000-S30704505-092-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378219-1305463843/watcher-3378219-1305463843 -o /tmp/evaluation-result-3378219-1305463843/solver-3378219-1305463843 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378219-1305463843.cnf -seed 1835479921 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 15b86e0c1384321ea7bbac2466a811a7
RANDOM SEED=1835479921

node103.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      24795916 kB
Buffers:        327484 kB
Cached:        4194276 kB
SwapCached:     340184 kB
Active:        2633608 kB
Inactive:      5212572 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      24795916 kB
SwapTotal:    67111528 kB
SwapFree:     53501244 kB
Dirty:           33876 kB
Writeback:           0 kB
AnonPages:     2995140 kB
Mapped:          27092 kB
Slab:           164280 kB
PageTables:      76364 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 140321844 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72896 MiB
End job on node103 at 2011-05-15 15:09:42