Trace number 3385361

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 9393.62 1181.51

General information on the benchmark

Namerandom/large/
unif-k7-r85-v250-c21250-S894356234-063-UNKNOWN.cnf
MD5SUMc26a52ff3714cab8bc335a81bb8af922
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables250
Number of clauses21250
Sum of the clauses size148750
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 521250

Solver Data 

0.36/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.36/0.67	c INFO: solving HOME/instance-3385361-1305565764.cnf
0.36/0.67	c INFO: computing task features
0.54/0.81	c INFO: computed variable-clause graph statistics
0.54/0.81	c INFO: computed clause balance statistics
1.24/1.56	c INFO: computed variable balance statistics
1.54/1.86	c INFO: computed small-clause counts
1.54/1.86	c INFO: computed Horn-clause counts
1.54/1.89	c INFO: computed variable graph statistics
1.54/1.89	c INFO: collected features for HOME/instance-3385361-1305565764.cnf in 1.22 s
1.63/1.92	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.36)
1.63/1.95	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.13)
1.63/1.98	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.08)
1.73/2.01	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.07)
1.73/2.04	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.05)
1.73/2.08	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.04)
2.40/2.11	c INFO: running FH@0 for 97 with 4998 remaining (b = 0.03)
2.40/2.14	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.03)
801.15/102.20	c INFO: running adaptg2wsat2009++@0 for 97 with 4898 remaining (b = 0.02)
802.03/102.25	c INFO: running FH@0 for 97 with 4898 remaining (b = 0.02)
802.03/102.29	c INFO: running hybridGM3@0 for 194 with 4898 remaining (b = 0.04)
802.51/102.33	c INFO: running gnovelty+2@0 for 97 with 4898 remaining (b = 0.02)
802.51/102.37	c INFO: running hybridGM3@0 for 194 with 4898 remaining (b = 0.03)
802.72/102.41	c INFO: running hybridGM3@0 for 194 with 4898 remaining (b = 0.03)
802.72/102.46	c INFO: running TNM@0 for 1552 with 4898 remaining (b = 0.20)
802.82/102.51	c INFO: running adaptg2wsat2009++@0 for 97 with 4898 remaining (b = 0.01)
1593.47/201.54	c INFO: running FH@0 for 97 with 4799 remaining (b = 0.01)
1600.73/202.48	c INFO: running hybridGM3@0 for 194 with 4798 remaining (b = 0.02)
1602.14/202.65	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.01)
1603.53/202.82	c INFO: running adaptg2wsat2009++@0 for 97 with 4797 remaining (b = 0.01)
2369.73/298.91	c INFO: running hybridGM3@100 for 97 with 4701 remaining (b = 0.01)
2369.73/298.96	c INFO: running hybridGM3@99 for 97 with 4701 remaining (b = 0.01)
2376.89/299.85	c INFO: running adaptg2wsat2009++@0 for 97 with 4700 remaining (b = 0.01)
2377.67/299.92	c INFO: running adaptg2wsat2009++@100 for 97 with 4700 remaining (b = 0.01)
2378.46/300.09	c INFO: running hybridGM3@0 for 194 with 4700 remaining (b = 0.02)
2416.68/304.85	c INFO: running hybridGM3@99 for 97 with 4695 remaining (b = 0.01)
3128.37/394.18	c INFO: running hybridGM3@296 for 97 with 4606 remaining (b = 0.01)
3151.55/397.09	c INFO: running hybridGM3@100 for 97 with 4603 remaining (b = 0.01)
3152.33/397.14	c INFO: running TNM@0 for 1455 with 4603 remaining (b = 0.14)
3152.93/397.20	c INFO: running adaptg2wsat2009++@100 for 97 with 4603 remaining (b = 0.01)
3176.06/400.11	c INFO: running hybridGM3@193 for 97 with 4600 remaining (b = 0.01)
3176.86/400.24	c INFO: running gnovelty+2@0 for 97 with 4600 remaining (b = 0.01)
3920.42/493.50	c INFO: running hybridGM3@0 for 194 with 4507 remaining (b = 0.02)
3935.56/495.46	c INFO: running iPAWS@0 for 97 with 4505 remaining (b = 0.01)
3945.07/496.64	c INFO: running hybridGM3@0 for 194 with 4504 remaining (b = 0.02)
3959.41/498.40	c INFO: running FH@0 for 291 with 4502 remaining (b = 0.02)
3976.20/500.56	c INFO: running adaptg2wsat2009++@99 for 97 with 4500 remaining (b = 0.01)
3982.41/501.39	c INFO: running hybridGM3@294 for 97 with 4499 remaining (b = 0.01)
4725.93/594.77	c INFO: running hybridGM3@0 for 194 with 4405 remaining (b = 0.01)
4741.05/596.67	c INFO: running hybridGM3@393 for 97 with 4404 remaining (b = 0.01)
4790.44/602.90	c INFO: running adaptg2wsat2009++@97 for 97 with 4397 remaining (b = 0.01)
5484.48/690.10	c INFO: running FH@100 for 291 with 4310 remaining (b = 0.02)
5499.64/691.95	c INFO: running hybridGM3@101 for 97 with 4308 remaining (b = 0.01)
5565.62/700.21	c INFO: running TNM@0 for 1552 with 4300 remaining (b = 0.12)
5597.47/704.29	c INFO: running adaptg2wsat2009++@293 for 291 with 4296 remaining (b = 0.02)
6265.26/788.15	c INFO: running iPAWS@0 for 97 with 4212 remaining (b = 0.01)
6297.86/792.25	c INFO: running hybridGM3@101 for 97 with 4208 remaining (b = 0.01)
6323.32/795.46	c INFO: running hybridGM3@94 for 97 with 4205 remaining (b = 0.01)
7063.61/888.45	c INFO: running gnovelty+2@0 for 97 with 4112 remaining (b = 0.01)
7112.15/894.59	c INFO: running hybridGM3@297 for 194 with 4106 remaining (b = 0.02)
7121.67/895.75	c INFO: running FH@0 for 97 with 4104 remaining (b = 0.01)
7823.73/983.90	c INFO: running hybridGM3@0 for 194 with 4016 remaining (b = 0.01)
7904.08/994.03	c INFO: running NCVWr@0 for 485 with 4006 remaining (b = 0.03)
7910.45/994.81	c INFO: running FH@195 for 194 with 4005 remaining (b = 0.01)
7912.84/995.11	c INFO: running FH@293 for 97 with 4005 remaining (b = 0.01)
8660.48/1089.17	c INFO: running hybridGM3@397 for 97 with 3911 remaining (b = 0.01)
8718.52/1096.41	c INFO: running iPAWS@0 for 97 with 3904 remaining (b = 0.01)
9387.09/1180.53	c INFO: running hybridGM3@0 for 194 with 3820 remaining (b = 0.01)
9393.62/1181.47	c INFO: running iPAWS@0 for 97 with 3819 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385361-1305565764/watcher-3385361-1305565764 -o /tmp/evaluation-result-3385361-1305565764/solver-3385361-1305565764 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385361-1305565764.cnf -seed 825455913 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.25 7.52 4.62 2/295 18642
/proc/meminfo: memFree=25693768/32951124 swapFree=67083432/67111528
[pid=18642] ppid=18640 vsize=15760 CPUtime=0 cores=0-7
/proc/18642/stat : 18642 (python) S 18640 18642 18357 0 -1 4202496 844 0 0 0 0 0 0 0 20 0 1 0 746806980 16138240 727 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020393984 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/18642/statm: 3940 727 334 364 0 427 0

[startup+0.045146 s]
/proc/loadavg: 8.25 7.52 4.62 2/295 18642
/proc/meminfo: memFree=25693768/32951124 swapFree=67083432/67111528
[pid=18642] ppid=18640 vsize=18880 CPUtime=0.02 cores=0-7
/proc/18642/stat : 18642 (python) R 18640 18642 18357 0 -1 4202496 1283 0 0 0 2 0 0 0 20 0 1 0 746806980 19333120 1136 33554432000 4194304 5683644 140733964813056 18446744073709551615 4865528 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/18642/statm: 4720 1136 421 364 0 689 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 18880

[startup+0.101148 s]
/proc/loadavg: 8.25 7.52 4.62 2/295 18642
/proc/meminfo: memFree=25693768/32951124 swapFree=67083432/67111528
[pid=18642] ppid=18640 vsize=89764 CPUtime=0.06 cores=0-7
/proc/18642/stat : 18642 (python) R 18640 18642 18357 0 -1 4202496 2671 0 0 0 5 1 0 0 20 0 1 0 746806980 91918336 2387 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020393984 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/18642/statm: 22441 2387 750 364 0 1621 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 89764

[startup+0.301148 s]
/proc/loadavg: 8.25 7.52 4.62 2/295 18642
/proc/meminfo: memFree=25693768/32951124 swapFree=67083432/67111528
[pid=18642] ppid=18640 vsize=135744 CPUtime=0.19 cores=0-7
/proc/18642/stat : 18642 (python) R 18640 18642 18357 0 -1 4202496 5962 933 1 0 14 5 0 0 17 0 1 0 746806980 139001856 4150 33554432000 4194304 5683644 140733964813056 18446744073709551615 4620076 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/18642/statm: 33936 4150 1032 364 0 3122 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135744

[startup+0.70107 s]
/proc/loadavg: 8.25 7.52 4.62 2/295 18642
/proc/meminfo: memFree=25693768/32951124 swapFree=67083432/67111528
[pid=18642] ppid=18640 vsize=279752 CPUtime=0.44 cores=0-7
/proc/18642/stat : 18642 (python) R 18640 18642 18357 0 -1 4202496 9705 933 1 0 34 10 0 0 16 0 1 0 746806980 286466048 7272 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020082597 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/18642/statm: 69938 7272 1761 364 0 5862 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 279752

[startup+1.50092 s]
/proc/loadavg: 8.25 7.52 4.62 2/296 18647
/proc/meminfo: memFree=25666604/32951124 swapFree=67083432/67111528
[pid=18642] ppid=18640 vsize=283912 CPUtime=1.24 cores=0-7
/proc/18642/stat : 18642 (python) R 18640 18642 18357 0 -1 4202496 11841 933 1 0 114 10 0 0 17 0 1 0 746806980 290725888 8363 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020063104 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/18642/statm: 70978 8363 1826 364 0 6902 0
Current children cumulated CPU time (s) 1.24
Current children cumulated vsize (KiB) 283912

[startup+3.10155 s]
/proc/loadavg: 7.67 7.41 4.60 8/314 18665
/proc/meminfo: memFree=25630368/32951124 swapFree=67083432/67111528
[pid=18642] ppid=18640 vsize=362116 CPUtime=1.87 cores=0-7
/proc/18642/stat : 18642 (python) S 18640 18642 18357 0 -1 4202496 18199 933 1 0 175 12 0 0 20 0 9 0 746806980 370806784 7496 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032222667 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/18642/statm: 90529 7496 1891 364 0 26426 0
[pid=18648] ppid=18642 vsize=279924 CPUtime=0 cores=0-7
/proc/18648/stat : 18648 (python) S 18642 18642 18357 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 746807173 286642176 5769 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416015 0 0 16781312 514 4273952884309065988 0 0 17 4 0 0 0
/proc/18648/statm: 69981 5769 355 364 0 5899 0
[pid=18650] ppid=18648 vsize=1018820 CPUtime=1.17 cores=0-7
/proc/18650/stat : 18650 (gnovelty+2) R 18648 18650 18650 0 -1 4202496 858 0 0 0 117 0 0 0 25 0 1 0 746807173 1043271680 595 33554432000 134512640 135663704 4286898944 18446744073709551615 134525276 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/18650/statm: 254705 595 113 282 0 254421 0
[pid=18651] ppid=18642 vsize=290820 CPUtime=0 cores=0-7
/proc/18651/stat : 18651 (python) S 18642 18642 18357 0 -1 4202560 738 0 0 0 0 0 0 0 18 0 1 0 746807176 297799680 5792 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416070 0 0 16781312 514 4334874074825064705 0 0 17 6 0 0 0
/proc/18651/statm: 72705 5792 349 364 0 8620 0
[pid=18653] ppid=18651 vsize=1018820 CPUtime=1.13 cores=0-7
/proc/18653/stat : 18653 (gnovelty+2) R 18651 18653 18653 0 -1 4202496 857 0 0 0 113 0 0 0 25 0 1 0 746807176 1043271680 595 33554432000 134512640 135663704 4291371712 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/18653/statm: 254705 595 113 282 0 254421 0
[pid=18654] ppid=18642 vsize=300592 CPUtime=0 cores=0-7
/proc/18654/stat : 18654 (python) S 18642 18642 18357 0 -1 4202560 745 0 0 0 0 0 0 0 18 0 1 0 746807179 307806208 5928 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416070 0 0 16781312 514 16003511593342074879 0 0 17 6 0 0 0
/proc/18654/statm: 75148 5928 347 364 0 11060 0
[pid=18656] ppid=18654 vsize=419160 CPUtime=1.1 cores=0-7
/proc/18656/stat : 18656 (adaptg2wsat2009) R 18654 18656 18656 0 -1 4202496 1260 0 0 0 110 0 0 0 25 0 1 0 746807180 429219840 991 33554432000 134512640 135107411 4289508736 18446744073709551615 134523457 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/18656/statm: 104790 991 49 146 0 103666 0
[pid=18657] ppid=18642 vsize=310848 CPUtime=0 cores=0-7
/proc/18657/stat : 18657 (python) S 18642 18642 18357 0 -1 4202560 746 0 0 0 0 0 0 0 18 0 1 0 746807182 318308352 5933 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416070 0 0 16781312 514 2583228631597711359 0 0 17 4 0 0 0
/proc/18657/statm: 77712 5933 347 364 0 13621 0
[pid=18659] ppid=18657 vsize=419160 CPUtime=1.06 cores=0-7
/proc/18659/stat : 18659 (adaptg2wsat2009) R 18657 18659 18659 0 -1 4202496 1257 0 0 0 106 0 0 0 24 0 1 0 746807183 429219840 992 33554432000 134512640 135107411 4291264688 18446744073709551615 134523319 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/18659/statm: 104790 992 49 146 0 103666 0
[pid=18660] ppid=18642 vsize=321104 CPUtime=0 cores=0-7
/proc/18660/stat : 18660 (python) S 18642 18642 18357 0 -1 4202560 758 0 0 0 0 0 0 0 18 0 1 0 746807185 328810496 5937 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416070 0 0 16781312 514 3070981060650074111 0 0 17 6 0 0 0
/proc/18660/statm: 80276 5937 347 364 0 16182 0
[pid=18662] ppid=18660 vsize=419160 CPUtime=1.04 cores=0-7
/proc/18662/stat : 18662 (adaptg2wsat2009) R 18660 18662 18662 0 -1 4202496 1259 0 0 0 104 0 0 0 25 0 1 0 746807186 429219840 991 33554432000 134512640 135107411 4294373200 18446744073709551615 134523424 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/18662/statm: 104790 991 49 146 0 103666 0
[pid=18663] ppid=18642 vsize=331360 CPUtime=0 cores=0-7
/proc/18663/stat : 18663 (python) S 18642 18642 18357 0 -1 4202560 760 0 0 0 0 0 0 0 19 0 1 0 746807189 339312640 5941 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416070 0 0 16781312 514 10022030074653442047 0 0 17 2 0 0 0
/proc/18663/statm: 82840 5941 347 364 0 18743 0
[pid=18665] ppid=18663 vsize=1018820 CPUtime=1.01 cores=0-7
/proc/18665/stat : 18665 (gnovelty+2) R 18663 18665 18665 0 -1 4202496 860 0 0 0 101 0 0 0 23 0 1 0 746807189 1043271680 595 33554432000 134512640 135663704 4290938112 18446744073709551615 134524509 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/18665/statm: 254705 595 113 282 0 254421 0
[pid=18642/tid=18649] ppid=18640 vsize=362116 CPUtime=0 cores=0-7
/proc/18642/task/18649/stat : 18649 (python) S 18640 18642 18357 0 -1 4202560 26 933 0 0 0 0 0 0 18 0 9 0 746807173 370806784 7496 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 1 0 0 0
[pid=18642/tid=18652] ppid=18640 vsize=362116 CPUtime=0 cores=0-7
/proc/18642/task/18652/stat : 18652 (python) S 18640 18642 18357 0 -1 4202560 25 933 0 0 0 0 0 0 22 0 9 0 746807176 370806784 7496 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 0 0 0 0
[pid=18642/tid=18655] ppid=18640 vsize=362116 CPUtime=0 cores=0-7
/proc/18642/task/18655/stat : 18655 (python) S 18640 18642 18357 0 -1 4202560 24 933 0 0 0 0 0 0 24 0 9 0 746807179 370806784 7496 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 0 0 0 0
[pid=18642/tid=18658] ppid=18640 vsize=362116 CPUtime=0 cores=0-7
/proc/18642/task/18658/stat : 18658 (python) S 18640 18642 18357 0 -1 4202560 23 933 0 0 0 0 0 0 20 0 9 0 746807182 370806784 7496 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 6 0 0 0
[pid=18642/tid=18661] ppid=18640 vsize=362116 CPUtime=0 cores=0-7
/proc/18642/task/18661/stat : 18661 (python) S 18640 18642 18357 0 -1 4202560 23 933 0 0 0 0 0 0 22 0 9 0 746807186 370806784 7496 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 6 0 0 0
[pid=18642/tid=18664] ppid=18640 vsize=362116 CPUtime=0 cores=0-7
/proc/18642/task/18664/stat : 18664 (python) S 18640 18642 18357 0 -1 4202560 23 933 0 0 0 0 0 0 20 0 9 0 746807189 370806784 7496 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 8.38
Current children cumulated vsize (KiB) 6510704

[startup+6.30096 s]
/proc/loadavg: 7.67 7.41 4.60 9/320 18671
/proc/meminfo: memFree=25610496/32951124 swapFree=67083432/67111528
[pid=18642] ppid=18640 vsize=362116 CPUtime=1.87 cores=0-7
/proc/18642/stat : 18642 (python) S 18640 18642 18357 0 -1 4202496 18199 933 1 0 175 12 0 0 20 0 9 0 746806980 370806784 7496 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032222667 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/18642/statm: 90529 7496 1891 364 0 26426 0
[pid=18648] ppid=18642 vsize=279924 CPUtime=0 cores=0-7
/proc/18648/stat : 18648 (python) S 18642 18642 18357 0 -1 4202560 897 0 0 0 0 0 0 0 18 0 1 0 746807173 286642176 5792 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416015 0 0 16781312 514 4273952884309065988 0 0 17 4 0 0 0
/proc/18648/statm: 69981 5792 378 364 0 5899 0
[pid=18650] ppid=18648 vsize=1018820 CPUtime=4.37 cores=0-7
/proc/18650/stat : 18650 (gnovelty+2) R 18648 18650 18650 0 -1 4202496 858 0 0 0 437 0 0 0 25 0 1 0 746807173 1043271680 595 33554432000 134512640 135663704 4286898944 18446744073709551615 134525276 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/18650/statm: 254705 595 113 282 0 254421 0
[pid=18651] ppid=18642 vsize=290820 CPUtime=0 cores=0-7
/proc/18651/stat : 18651 (python) S 18642 18642 18357 0 -1 4202560 897 0 0 0 0 0 0 0 18 0 1 0 746807176 297799680 5815 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416070 0 0 16781312 514 4334874074825064705 0 0 17 6 0 0 0
/proc/18651/statm: 72705 5815 372 364 0 8620 0
[pid=18653] ppid=18651 vsize=1018820 CPUtime=4.33 cores=0-7
/proc/18653/stat : 18653 (gnovelty+2) R 18651 18653 18653 0 -1 4202496 857 0 0 0 433 0 0 0 25 0 1 0 746807176 1043271680 595 33554432000 134512640 135663704 4291371712 18446744073709551615 134545646 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/18653/statm: 254705 595 113 282 0 254421 0
[pid=18654] ppid=18642 vsize=300592 CPUtime=0 cores=0-7
/proc/18654/stat : 18654 (python) S 18642 18642 18357 0 -1 4202560 904 0 0 0 0 0 0 0 18 0 1 0 746807179 307806208 5951 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416070 0 0 16781312 514 16003511593342042369 0 0 17 6 0 0 0
/proc/18654/statm: 75148 5951 370 364 0 11060 0
[pid=18656] ppid=18654 vsize=419428 CPUtime=4.3 cores=0-7
/proc/18656/stat : 18656 (adaptg2wsat2009) R 18654 18656 18656 0 -1 4202496 1445 0 0 0 430 0 0 0 25 0 1 0 746807180 429494272 1176 33554432000 134512640 135107411 4289508736 18446744073709551615 134531435 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/18656/statm: 104857 1176 59 146 0 103733 0
[pid=18657] ppid=18642 vsize=310848 CPUtime=0 cores=0-7
/proc/18657/stat : 18657 (python) S 18642 18642 18357 0 -1 4202560 923 0 0 0 0 0 0 0 18 0 1 0 746807182 318308352 5956 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416070 0 0 16781312 514 2583228631597678850 0 0 17 4 0 0 0
/proc/18657/statm: 77712 5956 370 364 0 13621 0
[pid=18659] ppid=18657 vsize=419428 CPUtime=4.23 cores=0-7
/proc/18659/stat : 18659 (adaptg2wsat2009) R 18657 18659 18659 0 -1 4202496 1442 0 0 0 423 0 0 0 25 0 1 0 746807183 429494272 1177 33554432000 134512640 135107411 4291264688 18446744073709551615 134531808 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/18659/statm: 104857 1177 59 146 0 103733 0
[pid=18660] ppid=18642 vsize=321104 CPUtime=0 cores=0-7
/proc/18660/stat : 18660 (python) S 18642 18642 18357 0 -1 4202560 932 0 0 0 0 0 0 0 18 0 1 0 746807185 328810496 5960 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416070 0 0 16781312 514 3070981060650041601 0 0 17 6 0 0 0
/proc/18660/statm: 80276 5960 370 364 0 16182 0
[pid=18662] ppid=18660 vsize=419428 CPUtime=4.2 cores=0-7
/proc/18662/stat : 18662 (adaptg2wsat2009) R 18660 18662 18662 0 -1 4202496 1444 0 0 0 420 0 0 0 25 0 1 0 746807186 429494272 1176 33554432000 134512640 135107411 4294373200 18446744073709551615 134531537 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/18662/statm: 104857 1176 59 146 0 103733 0
[pid=18663] ppid=18642 vsize=331360 CPUtime=0 cores=0-7
/proc/18663/stat : 18663 (python) S 18642 18642 18357 0 -1 4202560 938 0 0 0 0 0 0 0 18 0 1 0 746807189 339312640 5964 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416070 0 0 16781312 514 10022030074653409538 0 0 17 2 0 0 0
/proc/18663/statm: 82840 5964 370 364 0 18743 0
[pid=18665] ppid=18663 vsize=1018820 CPUtime=4.21 cores=0-7
/proc/18665/stat : 18665 (gnovelty+2) R 18663 18665 18665 0 -1 4202496 860 0 0 0 421 0 0 0 25 0 1 0 746807189 1043271680 595 33554432000 134512640 135663704 4290938112 18446744073709551615 134525255 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/18665/statm: 254705 595 113 282 0 254421 0
[pid=18666] ppid=18642 vsize=341796 CPUtime=0 cores=0-7
/proc/18666/stat : 18666 (python) S 18642 18642 18357 0 -1 4202560 936 0 0 0 0 0 0 0 18 0 1 0 746807192 349999104 5970 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416070 0 0 16781312 514 10022026226362712321 0 0 17 2 0 0 0
/proc/18666/statm: 85449 5970 372 364 0 21349 0
[pid=18668] ppid=18666 vsize=249196 CPUtime=4.14 cores=0-7
/proc/18668/stat : 18668 (FH) R 18666 18668 18668 0 -1 4202496 1324 0 0 0 414 0 0 0 25 0 1 0 746807192 255176704 1058 33554432000 134512640 134936988 4291001680 18446744073709551615 134554976 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/18668/statm: 62299 1058 41 104 0 61803 0
[pid=18669] ppid=18642 vsize=351872 CPUtime=0 cores=0-7
/proc/18669/stat : 18669 (python) S 18642 18642 18357 0 -1 4202560 984 0 0 0 0 0 0 0 18 0 1 0 746807195 360316928 5978 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416070 0 0 16781312 514 10022022378072015105 0 0 17 4 0 0 0
/proc/18669/statm: 87968 5978 376 364 0 23865 0
[pid=18671] ppid=18669 vsize=419428 CPUtime=4.1 cores=0-7
/proc/18671/stat : 18671 (adaptg2wsat2009) R 18669 18671 18671 0 -1 4202496 1450 0 0 0 410 0 0 0 25 0 1 0 746807195 429494272 1176 33554432000 134512640 135107411 4294735552 18446744073709551615 134531232 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/18671/statm: 104857 1176 59 146 0 103733 0
[pid=18642/tid=18649] ppid=18640 vsize=362116 CPUtime=0 cores=0-7
/proc/18642/task/18649/stat : 18649 (python) S 18640 18642 18357 0 -1 4202560 26 933 0 0 0 0 0 0 18 0 9 0 746807173 370806784 7496 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 1 0 0 0
[pid=18642/tid=18652] ppid=18640 vsize=362116 CPUtime=0 cores=0-7
/proc/18642/task/18652/stat : 18652 (python) S 18640 18642 18357 0 -1 4202560 25 933 0 0 0 0 0 0 22 0 9 0 746807176 370806784 7496 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 0 0 0 0
[pid=18642/tid=18655] ppid=18640 vsize=362116 CPUtime=0 cores=0-7
/proc/18642/task/18655/stat : 18655 (python) S 18640 18642 18357 0 -1 4202560 24 933 0 0 0 0 0 0 24 0 9 0 746807179 370806784 7496 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 0 0 0 0
[pid=18642/tid=18658] ppid=18640 vsize=362116 CPUtime=0 cores=0-7
/proc/18642/task/18658/stat : 18658 (python) S 18640 18642 18357 0 -1 4202560 23 933 0 0 0 0 0 0 20 0 9 0 746807182 370806784 7496 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 6 0 0 0
[pid=18642/tid=18661] ppid=18640 vsize=362116 CPUtime=0 cores=0-7
/proc/18642/task/18661/stat : 18661 (python) S 18640 18642 18357 0 -1 4202560 23 933 0 0 0 0 0 0 22 0 9 0 746807186 370806784 7496 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 6 0 0 0
[pid=18642/tid=18664] ppid=18640 vsize=362116 CPUtime=0 cores=0-7
/proc/18642/task/18664/stat : 18664 (python) S 18640 18642 18357 0 -1 4202560 23 933 0 0 0 0 0 0 20 0 9 0 746807189 370806784 7496 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 2 0 0 0
[pid=18642/tid=18667] ppid=18640 vsize=362116 CPUtime=0 cores=0-7
/proc/18642/task/18667/stat : 18667 (python) S 18640 18642 18357 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 9 0 746807192 370806784 7496 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 2 0 0 0
[pid=18642/tid=18670] ppid=18640 vsize=362116 CPUtime=0 cores=0-7
/proc/18642/task/18670/stat : 18670 (python) S 18640 18642 18357 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 9 0 746807195 370806784 7496 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 4 0 0 0

################
# More data... #
################

[pid=18741] ppid=18642 vsize=561304 CPUtime=0.74 cores=0-7
/proc/18741/stat : 18741 (python) S 18642 18642 18357 0 -1 4202560 1183 0 0 0 37 37 0 0 18 0 2 0 746856331 574775296 6538 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/18741/statm: 140326 6538 406 364 0 76181 0
[pid=18743] ppid=18741 vsize=268424 CPUtime=296.14 cores=0-7
/proc/18743/stat : 18743 (hybridGM3) T 18741 18743 18743 0 -1 4202496 5290 0 0 0 29614 0 0 0 25 0 1 0 746856332 274866176 5008 33554432000 134512640 135204385 4292853200 18446744073709551615 134538041 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/18743/statm: 67106 5008 63 169 0 66936 0
[pid=18741/tid=18761] ppid=18642 vsize=561304 CPUtime=0 cores=0-7
/proc/18741/task/18761/stat : 18761 (python) S 18642 18642 18357 0 -1 4202560 95 0 0 0 0 0 0 0 18 0 2 0 746875985 574775296 6538 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 514 18446604444696867264 0 0 -1 4 0 0 0
[pid=18744] ppid=18642 vsize=571560 CPUtime=0.25 cores=0-7
/proc/18744/stat : 18744 (python) S 18642 18642 18357 0 -1 4202560 1178 0 0 0 13 12 0 0 19 0 2 0 746856526 585277440 6556 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/18744/statm: 142890 6556 407 364 0 78742 0
[pid=18746] ppid=18744 vsize=335996 CPUtime=99 cores=0-7
/proc/18746/stat : 18746 (iPAWS) T 18744 18746 18746 0 -1 4202496 1707 0 0 0 9900 0 0 0 25 0 1 0 746856527 344059904 1428 33554432000 134512640 135114423 4288459248 18446744073709551615 134516967 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/18746/statm: 83999 1428 59 147 0 83851 0
[pid=18744/tid=18755] ppid=18642 vsize=571560 CPUtime=0 cores=0-7
/proc/18744/task/18755/stat : 18755 (python) S 18642 18642 18357 0 -1 4202560 95 0 0 0 0 0 0 0 19 0 2 0 746866452 585277440 6556 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 514 18446604453495843520 0 0 -1 0 0 0 0
[pid=18748] ppid=18642 vsize=581816 CPUtime=1.55 cores=0-7
/proc/18748/stat : 18748 (python) S 18642 18642 18357 0 -1 4202560 1197 0 0 0 79 76 0 0 18 0 2 0 746856645 595779584 6508 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416070 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/18748/statm: 145454 6508 407 364 0 81303 0
[pid=18750] ppid=18748 vsize=268424 CPUtime=577.76 cores=0-7
/proc/18750/stat : 18750 (hybridGM3) R 18748 18750 18750 0 -1 4202496 5290 0 0 0 57775 1 0 0 25 0 1 0 746856645 274866176 5008 33554432000 134512640 135204385 4292611264 18446744073709551615 134538284 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/18750/statm: 67106 5008 63 169 0 66936 0
[pid=18748/tid=18766] ppid=18642 vsize=581816 CPUtime=0 cores=0-7
/proc/18748/task/18766/stat : 18766 (python) S 18642 18642 18357 0 -1 4202560 97 0 0 0 0 0 0 0 18 0 2 0 746877403 595779584 6508 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 514 18446604441290357632 0 0 -1 7 0 0 0
[pid=18756] ppid=18642 vsize=592724 CPUtime=0.76 cores=0-7
/proc/18756/stat : 18756 (python) S 18642 18642 18357 0 -1 4202560 1288 0 0 0 40 36 0 0 18 0 2 0 746866458 606949376 6461 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18756/statm: 148181 6461 402 364 0 84027 0
[pid=18758] ppid=18756 vsize=268420 CPUtime=295.84 cores=0-7
/proc/18758/stat : 18758 (hybridGM3) T 18756 18758 18758 0 -1 4202496 5298 0 0 0 29583 1 0 0 25 0 1 0 746866459 274862080 5009 33554432000 134512640 135204385 4292377344 18446744073709551615 134538432 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/18758/statm: 67105 5009 63 169 0 66935 0
[pid=18756/tid=18772] ppid=18642 vsize=592724 CPUtime=0 cores=0-7
/proc/18756/task/18772/stat : 18772 (python) S 18642 18642 18357 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 746886520 606949376 6461 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 514 18446604453175370816 0 0 -1 1 0 0 0
[pid=18762] ppid=18642 vsize=602328 CPUtime=1.2 cores=0-7
/proc/18762/stat : 18762 (python) S 18642 18642 18357 0 -1 4202560 975 0 0 0 64 56 0 0 18 0 1 0 746877002 616783872 6586 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416070 0 0 16781312 514 17115560947914604804 0 0 17 6 0 0 0
/proc/18762/statm: 150582 6586 373 364 0 86425 0
[pid=18764] ppid=18762 vsize=457664 CPUtime=474.24 cores=0-7
/proc/18764/stat : 18764 (TNM) R 18762 18764 18764 0 -1 4202496 1318 0 0 0 47422 2 0 0 25 0 1 0 746877002 468647936 1036 33554432000 134512640 135008748 4289678016 18446744073709551615 134527773 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/18764/statm: 114416 1036 46 122 0 113316 0
[pid=18769] ppid=18642 vsize=612584 CPUtime=0.24 cores=0-7
/proc/18769/stat : 18769 (python) S 18642 18642 18357 0 -1 4202560 1193 0 0 0 13 11 0 0 19 0 2 0 746885796 627286016 6692 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/18769/statm: 153146 6692 401 364 0 88986 0
[pid=18771] ppid=18769 vsize=335996 CPUtime=100.04 cores=0-7
/proc/18771/stat : 18771 (iPAWS) T 18769 18771 18771 0 -1 4202496 1710 0 0 0 10004 0 0 0 25 0 1 0 746885796 344059904 1429 33554432000 134512640 135114423 4293346016 18446744073709551615 134514774 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/18771/statm: 83999 1429 59 147 0 83851 0
[pid=18769/tid=18775] ppid=18642 vsize=612584 CPUtime=0 cores=0-7
/proc/18769/task/18775/stat : 18775 (python) S 18642 18642 18357 0 -1 4202560 99 0 0 0 0 0 0 0 19 0 2 0 746895820 627286016 6692 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 514 18446604468194874368 0 0 -1 0 0 0 0
[pid=18776] ppid=18642 vsize=622840 CPUtime=0.26 cores=0-7
/proc/18776/stat : 18776 (python) S 18642 18642 18357 0 -1 4202560 1202 0 0 0 15 11 0 0 19 0 2 0 746895826 637788160 6634 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18776/statm: 155710 6634 401 364 0 91547 0
[pid=18778] ppid=18776 vsize=1018820 CPUtime=101.5 cores=0-7
/proc/18778/stat : 18778 (gnovelty+2) T 18776 18778 18778 0 -1 4202496 878 0 0 0 10150 0 0 0 25 0 1 0 746895827 1043271680 595 33554432000 134512640 135663704 4294624480 18446744073709551615 134524713 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/18778/statm: 254705 595 113 282 0 254421 0
[pid=18776/tid=18791] ppid=18642 vsize=622840 CPUtime=0 cores=0-7
/proc/18776/task/18791/stat : 18791 (python) S 18642 18642 18357 0 -1 4202560 103 0 0 0 0 0 0 0 19 0 2 0 746906456 637788160 6634 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 514 18446604452120545088 0 0 -1 7 0 0 0
[pid=18780] ppid=18642 vsize=633096 CPUtime=0.5 cores=0-7
/proc/18780/stat : 18780 (python) S 18642 18642 18357 0 -1 4202560 1245 0 0 0 25 25 0 0 18 0 2 0 746896556 648290304 6689 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18780/statm: 158274 6689 407 364 0 94108 0
[pid=18782] ppid=18780 vsize=249188 CPUtime=199.43 cores=0-7
/proc/18782/stat : 18782 (FH) T 18780 18782 18782 0 -1 4202496 1356 0 0 0 19943 0 0 0 25 0 1 0 746896557 255168512 1058 33554432000 134512640 134936988 4286952240 18446744073709551615 134554064 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/18782/statm: 62297 1058 41 104 0 61801 0
[pid=18780/tid=18792] ppid=18642 vsize=633096 CPUtime=0 cores=0-7
/proc/18780/task/18792/stat : 18792 (python) S 18642 18642 18357 0 -1 4202560 101 0 0 0 0 0 0 0 18 0 2 0 746906486 648290304 6689 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 514 18446604443138520064 0 0 -1 1 0 0 0
[pid=18785] ppid=18642 vsize=643352 CPUtime=0.49 cores=0-7
/proc/18785/stat : 18785 (python) S 18642 18642 18357 0 -1 4202560 1238 0 0 0 23 26 0 0 19 0 2 0 746905371 658792448 6751 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/18785/statm: 160838 6751 407 364 0 96669 0
[pid=18787] ppid=18785 vsize=268420 CPUtime=196.49 cores=0-7
/proc/18787/stat : 18787 (hybridGM3) T 18785 18787 18787 0 -1 4202496 5297 0 0 0 19649 0 0 0 25 0 1 0 746905372 274862080 5009 33554432000 134512640 135204385 4292143840 18446744073709551615 134538264 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/18787/statm: 67105 5009 63 169 0 66935 0
[pid=18785/tid=18801] ppid=18642 vsize=643352 CPUtime=0 cores=0-7
/proc/18785/task/18801/stat : 18801 (python) S 18642 18642 18357 0 -1 4202560 99 0 0 0 0 0 0 0 19 0 2 0 746925028 658792448 6751 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 514 18446604441115607424 0 0 -1 6 0 0 0
[pid=18788] ppid=18642 vsize=653608 CPUtime=0.47 cores=0-7
/proc/18788/stat : 18788 (python) S 18642 18642 18357 0 -1 4202560 1045 0 0 0 26 21 0 0 18 0 1 0 746906384 669294592 6754 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416070 0 0 16781312 514 17115560947914604801 0 0 17 1 0 0 0
/proc/18788/statm: 163402 6754 379 364 0 99230 0
[pid=18790] ppid=18788 vsize=419068 CPUtime=186.32 cores=0-7
/proc/18790/stat : 18790 (NCVWr) R 18788 18790 18790 0 -1 4202496 1367 0 0 0 18632 0 0 0 25 0 1 0 746906384 429125632 1077 33554432000 134512640 135113558 4292754624 18446744073709551615 134533486 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/18790/statm: 104767 1077 58 147 0 103642 0
[pid=18796] ppid=18642 vsize=663864 CPUtime=0.21 cores=0-7
/proc/18796/stat : 18796 (python) S 18642 18642 18357 0 -1 4202560 1059 0 0 0 12 9 0 0 18 0 1 0 746916622 679796736 6646 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416070 0 0 16781312 514 17115560947914604801 0 0 17 4 0 0 0
/proc/18796/statm: 165966 6646 379 364 0 101791 0
[pid=18798] ppid=18796 vsize=335996 CPUtime=84.67 cores=0-7
/proc/18798/stat : 18798 (iPAWS) R 18796 18798 18798 0 -1 4202496 1718 0 0 0 8467 0 0 0 25 0 1 0 746916622 344059904 1429 33554432000 134512640 135114423 4289579328 18446744073709551615 134516971 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/18798/statm: 83999 1429 59 147 0 83851 0
[pid=18802] ppid=18642 vsize=674120 CPUtime=0 cores=0-7
/proc/18802/stat : 18802 (python) S 18642 18642 18357 0 -1 4202560 960 0 0 0 0 0 0 0 19 0 1 0 746925034 690298880 6722 33554432000 4194304 5683644 140733964813056 18446744073709551615 260020416070 0 0 16781312 514 4273960580890492927 0 0 17 4 0 0 0
/proc/18802/statm: 168530 6722 356 364 0 104352 0
[pid=18804] ppid=18802 vsize=268424 CPUtime=0.77 cores=0-7
/proc/18804/stat : 18804 (hybridGM3) R 18802 18804 18804 0 -1 4202496 5301 0 0 0 77 0 0 0 22 0 1 0 746925035 274866176 5010 33554432000 134512640 135204385 4288418864 18446744073709551615 134538041 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/18804/statm: 67106 5010 63 169 0 66936 0
[pid=18642/tid=18649] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18649/stat : 18649 (python) S 18640 18642 18357 0 -1 4202560 26 933 0 0 0 0 0 0 18 0 34 0 746807173 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 1 0 0 0
[pid=18642/tid=18652] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18652/stat : 18652 (python) S 18640 18642 18357 0 -1 4202560 25 933 0 0 0 0 0 0 22 0 34 0 746807176 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 0 0 0 0
[pid=18642/tid=18655] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18655/stat : 18655 (python) S 18640 18642 18357 0 -1 4202560 36 933 0 0 0 0 0 0 18 0 34 0 746807179 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 0 0 0 0
[pid=18642/tid=18658] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18658/stat : 18658 (python) S 18640 18642 18357 0 -1 4202560 29 933 0 0 0 0 0 0 18 0 34 0 746807182 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 6 0 0 0
[pid=18642/tid=18661] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18661/stat : 18661 (python) S 18640 18642 18357 0 -1 4202560 28 933 0 0 0 0 0 0 18 0 34 0 746807186 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 6 0 0 0
[pid=18642/tid=18664] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18664/stat : 18664 (python) S 18640 18642 18357 0 -1 4202560 23 933 0 0 0 0 0 0 20 0 34 0 746807189 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 2 0 0 0
[pid=18642/tid=18667] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18667/stat : 18667 (python) S 18640 18642 18357 0 -1 4202560 27 933 0 0 0 0 0 0 18 0 34 0 746807192 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 2 0 0 0
[pid=18642/tid=18670] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18670/stat : 18670 (python) S 18640 18642 18357 0 -1 4202560 35 933 0 0 0 0 0 0 18 0 34 0 746807195 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 4 0 0 0
[pid=18642/tid=18678] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18678/stat : 18678 (python) S 18640 18642 18357 0 -1 4202560 45 933 0 0 0 0 0 0 18 0 34 0 746817201 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 1 0 0 0
[pid=18642/tid=18683] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18683/stat : 18683 (python) S 18640 18642 18357 0 -1 4202560 99 933 0 0 0 0 0 0 18 0 34 0 746817206 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 6 0 0 0
[pid=18642/tid=18687] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18687/stat : 18687 (python) S 18640 18642 18357 0 -1 4202560 53 933 0 0 0 0 0 0 18 0 34 0 746817210 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 0 0 0 0
[pid=18642/tid=18691] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18691/stat : 18691 (python) S 18640 18642 18357 0 -1 4202560 31 933 0 0 0 0 0 0 24 0 34 0 746817214 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 7 0 0 0
[pid=18642/tid=18696] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18696/stat : 18696 (python) S 18640 18642 18357 0 -1 4202560 33 933 0 0 0 0 0 0 15 0 34 0 746817218 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 4 0 0 0
[pid=18642/tid=18699] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18699/stat : 18699 (python) S 18640 18642 18357 0 -1 4202560 32 933 0 0 0 0 0 0 18 0 34 0 746817223 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 4 0 0 0
[pid=18642/tid=18702] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18702/stat : 18702 (python) S 18640 18642 18357 0 -1 4202560 21 933 0 0 0 0 0 0 19 0 34 0 746817227 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 6 0 0 0
[pid=18642/tid=18705] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18705/stat : 18705 (python) S 18640 18642 18357 0 -1 4202560 34 933 0 0 0 0 0 0 18 0 34 0 746817232 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 1 0 0 0
[pid=18642/tid=18711] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18711/stat : 18711 (python) S 18640 18642 18357 0 -1 4202560 29 933 0 0 0 0 0 0 20 0 34 0 746827135 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 1 0 0 0
[pid=18642/tid=18715] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18715/stat : 18715 (python) S 18640 18642 18357 0 -1 4202560 125 933 0 0 0 0 0 0 18 0 34 0 746827229 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 5 0 0 0
[pid=18642/tid=18726] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18726/stat : 18726 (python) S 18640 18642 18357 0 -1 4202560 39 933 0 0 0 0 0 0 20 0 34 0 746836990 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 6 0 0 0
[pid=18642/tid=18733] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18733/stat : 18733 (python) S 18640 18642 18357 0 -1 4202560 41 933 0 0 0 0 0 0 19 0 34 0 746846696 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 7 0 0 0
[pid=18642/tid=18736] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18736/stat : 18736 (python) S 18640 18642 18357 0 -1 4202560 46 933 0 0 0 0 0 0 19 0 34 0 746847005 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 6 0 0 0
[pid=18642/tid=18742] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18742/stat : 18742 (python) S 18640 18642 18357 0 -1 4202560 40 933 0 0 0 0 0 0 18 0 34 0 746856331 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 1 0 0 0
[pid=18642/tid=18745] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18745/stat : 18745 (python) S 18640 18642 18357 0 -1 4202560 32 933 0 0 0 0 0 0 19 0 34 0 746856527 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 1 0 0 0
[pid=18642/tid=18749] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18749/stat : 18749 (python) S 18640 18642 18357 0 -1 4202560 55 933 0 0 0 0 0 0 18 0 34 0 746856645 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 6 0 0 0
[pid=18642/tid=18757] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18757/stat : 18757 (python) S 18640 18642 18357 0 -1 4202560 34 933 0 0 0 0 0 0 18 0 34 0 746866458 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 1 0 0 0
[pid=18642/tid=18763] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18763/stat : 18763 (python) S 18640 18642 18357 0 -1 4202560 74 933 0 0 0 0 0 0 19 0 34 0 746877002 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 6 0 0 0
[pid=18642/tid=18770] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18770/stat : 18770 (python) S 18640 18642 18357 0 -1 4202560 39 933 0 0 0 0 0 0 19 0 34 0 746885796 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 7 0 0 0
[pid=18642/tid=18777] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18777/stat : 18777 (python) S 18640 18642 18357 0 -1 4202560 42 933 0 0 0 0 0 0 20 0 34 0 746895826 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 1 0 0 0
[pid=18642/tid=18781] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18781/stat : 18781 (python) S 18640 18642 18357 0 -1 4202560 41 933 0 0 0 0 0 0 18 0 34 0 746896557 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 4 0 0 0
[pid=18642/tid=18786] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18786/stat : 18786 (python) S 18640 18642 18357 0 -1 4202560 35 933 0 0 0 0 0 0 20 0 34 0 746905372 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 1 0 0 0
[pid=18642/tid=18789] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18789/stat : 18789 (python) S 18640 18642 18357 0 -1 4202560 32 933 0 0 0 0 0 0 19 0 34 0 746906384 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 3 0 0 0
[pid=18642/tid=18797] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18797/stat : 18797 (python) S 18640 18642 18357 0 -1 4202560 37 933 0 0 0 0 0 0 20 0 34 0 746916622 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 1 0 0 0
[pid=18642/tid=18803] ppid=18640 vsize=684364 CPUtime=0 cores=0-7
/proc/18642/task/18803/stat : 18803 (python) S 18640 18642 18357 0 -1 4202560 37 933 0 0 0 0 0 0 20 0 34 0 746925035 700788736 8312 33554432000 4194304 5683644 140733964813056 18446744073709551615 260032219569 0 0 16781312 2 18446604440629245248 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 9393.62
Current children cumulated vsize (KiB) 32175740

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 18642 and gives
#  childrusage.ru_utime.tv_sec=2923
#  childrusage.ru_utime.tv_usec=756521
#  childrusage.ru_stime.tv_sec=10
#  childrusage.ru_stime.tv_usec=520400
# CPU time returned by wait4() is 2934.28
# while last known CPU time is 9393.62
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1181.51
CPU time (s): 9393.62
CPU user time (s): 9382.22
CPU system time (s): 11.4
CPU usage (%): 795.053
Max. virtual memory (cumulated for all children) (KiB): 32175740

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2923.76
system time used= 10.5204
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 92206
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 13592
involuntary context switches= 26030

runsolver used 12.5641 second user time and 28.4597 second system time

The end

Launcher Data 

Begin job on node110 at 2011-05-16 19:09:24
IDJOB=3385361
IDBENCH=82626
IDSOLVER=1870
FILE ID=node110/3385361-1305565764
RUNJOBID= node110-1305564966-18371
PBS_JOBID= 13350140
Free space on /tmp= 73516 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k7-r85-v250-c21250-S894356234-063-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385361-1305565764/watcher-3385361-1305565764 -o /tmp/evaluation-result-3385361-1305565764/solver-3385361-1305565764 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385361-1305565764.cnf -seed 825455913 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= c26a52ff3714cab8bc335a81bb8af922
RANDOM SEED=825455913

node110.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25694048 kB
Buffers:        283292 kB
Cached:        3241280 kB
SwapCached:       1416 kB
Active:        3765720 kB
Inactive:      3163636 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25694048 kB
SwapTotal:    67111528 kB
SwapFree:     67083432 kB
Dirty:           32248 kB
Writeback:           0 kB
AnonPages:     3402928 kB
Mapped:          26636 kB
Slab:           236788 kB
PageTables:      23792 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 63402124 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73484 MiB
End job on node110 at 2011-05-16 19:29:08