Trace number 3321752

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 8703.62 1094.61

General information on the benchmark

Namerandom/large/
unif-k7-r85-v250-c21250-S381985918-018-UNKNOWN.cnf
MD5SUM13d193ddef22075908d36a7c0d4a2be1
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables250
Number of clauses21250
Sum of the clauses size148750
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 521250

Solver Data 

0.35/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.35/0.67	c INFO: solving HOME/instance-3321752-1304366611.cnf
0.35/0.67	c INFO: computing task features
0.52/0.81	c INFO: computed variable-clause graph statistics
0.52/0.82	c INFO: computed clause balance statistics
1.22/1.54	c INFO: computed variable balance statistics
1.52/1.84	c INFO: computed small-clause counts
1.52/1.84	c INFO: computed Horn-clause counts
1.52/1.86	c INFO: computed variable graph statistics
1.52/1.86	c INFO: collected features for HOME/instance-3321752-1304366611.cnf in 1.19 s
1.61/1.90	c INFO: running gnovelty+2@0 for 97 with 39998 remaining (b = 0.36)
1.61/1.94	c INFO: running gnovelty+2@0 for 97 with 39998 remaining (b = 0.13)
1.61/1.97	c INFO: running adaptg2wsat2009++@0 for 97 with 39998 remaining (b = 0.08)
1.71/2.01	c INFO: running adaptg2wsat2009++@0 for 97 with 39998 remaining (b = 0.07)
1.71/2.05	c INFO: running adaptg2wsat2009++@0 for 97 with 39998 remaining (b = 0.05)
1.71/2.09	c INFO: running gnovelty+2@0 for 97 with 39998 remaining (b = 0.04)
2.41/2.13	c INFO: running FH@0 for 97 with 39998 remaining (b = 0.03)
2.41/2.17	c INFO: running adaptg2wsat2009++@0 for 97 with 39998 remaining (b = 0.03)
801.21/102.17	c INFO: running adaptg2wsat2009++@0 for 97 with 39898 remaining (b = 0.02)
802.02/102.22	c INFO: running FH@0 for 97 with 39898 remaining (b = 0.02)
802.02/102.27	c INFO: running hybridGM3@0 for 194 with 39898 remaining (b = 0.04)
802.56/102.32	c INFO: running gnovelty+2@0 for 97 with 39898 remaining (b = 0.02)
802.56/102.38	c INFO: running hybridGM3@0 for 194 with 39898 remaining (b = 0.03)
802.84/102.44	c INFO: running hybridGM3@0 for 194 with 39898 remaining (b = 0.03)
803.03/102.50	c INFO: running TNM@0 for 1552 with 39898 remaining (b = 0.20)
803.03/102.56	c INFO: running adaptg2wsat2009++@0 for 97 with 39898 remaining (b = 0.01)
1600.93/202.47	c INFO: running FH@0 for 97 with 39798 remaining (b = 0.01)
1601.56/202.54	c INFO: running hybridGM3@0 for 194 with 39798 remaining (b = 0.02)
1602.25/202.63	c INFO: running adaptg2wsat2009++@0 for 97 with 39798 remaining (b = 0.01)
1603.48/202.84	c INFO: running adaptg2wsat2009++@0 for 97 with 39797 remaining (b = 0.01)
2353.32/296.84	c INFO: running hybridGM3@99 for 97 with 39703 remaining (b = 0.01)
2370.05/299.00	c INFO: running hybridGM3@99 for 97 with 39701 remaining (b = 0.01)
2378.00/299.90	c INFO: running adaptg2wsat2009++@0 for 97 with 39700 remaining (b = 0.01)
2378.00/299.99	c INFO: running hybridGM3@100 for 97 with 39700 remaining (b = 0.01)
2400.30/302.76	c INFO: running adaptg2wsat2009++@100 for 97 with 39697 remaining (b = 0.01)
2427.42/306.13	c INFO: running hybridGM3@0 for 194 with 39694 remaining (b = 0.02)
3130.43/394.31	c INFO: running hybridGM3@0 for 194 with 39606 remaining (b = 0.02)
3152.82/397.19	c INFO: running TNM@0 for 1552 with 39603 remaining (b = 0.15)
3160.72/398.12	c INFO: running hybridGM3@295 for 97 with 39602 remaining (b = 0.01)
3168.64/399.15	c INFO: running gnovelty+2@0 for 97 with 39601 remaining (b = 0.01)
3169.47/399.30	c INFO: running hybridGM3@99 for 97 with 39601 remaining (b = 0.01)
3175.77/400.04	c INFO: running adaptg2wsat2009++@100 for 97 with 39600 remaining (b = 0.01)
3951.30/497.33	c INFO: running FH@0 for 291 with 39503 remaining (b = 0.03)
3967.30/499.41	c INFO: running iPAWS@0 for 97 with 39501 remaining (b = 0.01)
3968.74/499.54	c INFO: running adaptg2wsat2009++@99 for 97 with 39501 remaining (b = 0.01)
3969.43/499.62	c INFO: running hybridGM3@0 for 194 with 39501 remaining (b = 0.02)
4041.96/508.72	c INFO: running hybridGM3@94 for 97 with 39492 remaining (b = 0.01)
4679.72/588.90	c INFO: running hybridGM3@97 for 97 with 39411 remaining (b = 0.01)
4743.39/596.83	c INFO: running hybridGM3@0 for 194 with 39403 remaining (b = 0.02)
4766.43/599.71	c INFO: running FH@100 for 291 with 39401 remaining (b = 0.02)
4800.68/604.01	c INFO: running iPAWS@0 for 97 with 39396 remaining (b = 0.01)
5486.94/690.23	c INFO: running hybridGM3@0 for 194 with 39310 remaining (b = 0.02)
5536.22/696.43	c INFO: running hybridGM3@0 for 194 with 39304 remaining (b = 0.01)
5591.14/703.37	c INFO: running TNM@0 for 1552 with 39297 remaining (b = 0.11)
6300.43/792.46	c INFO: running hybridGM3@99 for 97 with 39208 remaining (b = 0.01)
6305.26/793.05	c INFO: running adaptg2wsat2009++@97 for 97 with 39207 remaining (b = 0.01)
7051.91/886.83	c INFO: running FH@0 for 97 with 39113 remaining (b = 0.01)
7072.59/889.44	c INFO: running FH@195 for 194 with 39111 remaining (b = 0.01)
7079.77/890.37	c INFO: running adaptg2wsat2009++@293 for 291 with 39110 remaining (b = 0.02)
7101.25/893.03	c INFO: running hybridGM3@98 for 97 with 39107 remaining (b = 0.01)
7106.84/893.78	c INFO: running hybridGM3@394 for 97 with 39106 remaining (b = 0.01)
7850.34/987.17	c INFO: running NCVWr@0 for 485 with 39013 remaining (b = 0.03)
7865.41/989.10	c INFO: running FH@289 for 194 with 39011 remaining (b = 0.01)
7900.40/993.41	c INFO: running gnovelty+2@0 for 97 with 39007 remaining (b = 0.01)
8643.85/1086.98	c INFO: running FH@292 for 97 with 38913 remaining (b = 0.01)
8697.90/1093.78	c INFO: running hybridGM3@0 for 194 with 38906 remaining (b = 0.01)
8703.62/1094.55	c INFO: running iPAWS@0 for 97 with 38906 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321752-1304366611/watcher-3321752-1304366611 -o /tmp/evaluation-result-3321752-1304366611/solver-3321752-1304366611 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321752-1304366611.cnf -seed 353230720 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.08 2.79 1.08 1/261 21716
/proc/meminfo: memFree=23562732/32950940 swapFree=67111364/67111528
[pid=21716] ppid=21714 vsize=15380 CPUtime=0 cores=0-7
/proc/21716/stat : 21716 (python) S 21714 21716 21550 0 -1 4202496 820 0 0 0 0 0 0 0 17 0 1 0 626891921 15749120 704 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988590592 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/21716/statm: 3845 704 334 364 0 332 0

[startup+0.0155369 s]
/proc/loadavg: 7.08 2.79 1.08 1/261 21716
/proc/meminfo: memFree=23562732/32950940 swapFree=67111364/67111528
[pid=21716] ppid=21714 vsize=15908 CPUtime=0 cores=0-7
/proc/21716/stat : 21716 (python) S 21714 21716 21550 0 -1 4202496 945 0 0 0 0 0 0 0 17 0 1 0 626891921 16289792 821 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988590592 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/21716/statm: 3977 821 357 364 0 464 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15908

[startup+0.100413 s]
/proc/loadavg: 7.08 2.79 1.08 1/261 21716
/proc/meminfo: memFree=23562732/32950940 swapFree=67111364/67111528
[pid=21716] ppid=21714 vsize=85624 CPUtime=0.05 cores=0-7
/proc/21716/stat : 21716 (python) R 21714 21716 21550 0 -1 4202496 2631 0 0 0 4 1 0 0 17 0 1 0 626891921 87678976 2348 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988635783 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/21716/statm: 21405 2348 713 364 0 1622 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 85624

[startup+0.300468 s]
/proc/loadavg: 7.08 2.79 1.08 1/261 21716
/proc/meminfo: memFree=23562732/32950940 swapFree=67111364/67111528
[pid=21716] ppid=21714 vsize=135748 CPUtime=0.18 cores=0-7
/proc/21716/stat : 21716 (python) R 21714 21716 21550 0 -1 4202496 5968 931 1 0 13 5 0 0 17 0 1 0 626891921 139005952 4151 33554432000 4194304 5683644 140734622909168 18446744073709551615 4620076 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/21716/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 135748

[startup+0.700306 s]
/proc/loadavg: 7.08 2.79 1.08 1/261 21716
/proc/meminfo: memFree=23562732/32950940 swapFree=67111364/67111528
[pid=21716] ppid=21714 vsize=279512 CPUtime=0.42 cores=0-7
/proc/21716/stat : 21716 (python) R 21714 21716 21550 0 -1 4202496 9530 931 1 0 32 10 0 0 16 0 1 0 626891921 286220288 7208 33554432000 4194304 5683644 140734622909168 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/21716/statm: 69878 7208 1761 364 0 5802 0
Current children cumulated CPU time (s) 0.42
Current children cumulated vsize (KiB) 279512

[startup+1.50016 s]
/proc/loadavg: 7.08 2.79 1.08 2/262 21721
/proc/meminfo: memFree=23536188/32950940 swapFree=67111364/67111528
[pid=21716] ppid=21714 vsize=283916 CPUtime=1.22 cores=0-7
/proc/21716/stat : 21716 (python) R 21714 21716 21550 0 -1 4202496 11847 931 1 0 112 10 0 0 17 0 1 0 626891921 290729984 8364 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988251314 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/21716/statm: 70979 8364 1826 364 0 6903 0
Current children cumulated CPU time (s) 1.22
Current children cumulated vsize (KiB) 283916

[startup+3.10088 s]
/proc/loadavg: 7.08 2.79 1.08 8/280 21739
/proc/meminfo: memFree=23496932/32950940 swapFree=67111364/67111528
[pid=21716] ppid=21714 vsize=362100 CPUtime=1.88 cores=0-7
/proc/21716/stat : 21716 (python) S 21714 21716 21550 0 -1 4202496 18200 931 1 0 176 12 0 0 25 0 9 0 626891921 370790400 7492 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000419275 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/21716/statm: 90525 7492 1891 364 0 26422 0
[pid=21722] ppid=21716 vsize=279940 CPUtime=0 cores=0-7
/proc/21722/stat : 21722 (python) S 21716 21716 21550 0 -1 4202560 735 0 0 0 0 0 0 0 18 0 1 0 626892112 286658560 5773 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612623 0 0 16781312 514 2950478709414396162 0 0 17 5 0 0 0
/proc/21722/statm: 69985 5773 355 364 0 5903 0
[pid=21724] ppid=21722 vsize=1018820 CPUtime=1.18 cores=0-7
/proc/21724/stat : 21724 (gnovelty+2) R 21722 21724 21724 0 -1 4202496 859 0 0 0 118 0 0 0 22 0 1 0 626892112 1043271680 596 33554432000 134512640 135663704 4294328976 18446744073709551615 134524868 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/21724/statm: 254705 596 113 282 0 254421 0
[pid=21725] ppid=21716 vsize=290804 CPUtime=0 cores=0-7
/proc/21725/stat : 21725 (python) S 21716 21716 21550 0 -1 4202560 740 0 0 0 0 0 0 0 18 0 1 0 626892116 297783296 5788 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612678 0 0 16781312 514 4254904120235753730 0 0 17 4 0 0 0
/proc/21725/statm: 72701 5788 349 364 0 8616 0
[pid=21727] ppid=21725 vsize=1018820 CPUtime=1.15 cores=0-7
/proc/21727/stat : 21727 (gnovelty+2) R 21725 21727 21727 0 -1 4202496 858 0 0 0 115 0 0 0 25 0 1 0 626892116 1043271680 595 33554432000 134512640 135663704 4289991568 18446744073709551615 134525332 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/21727/statm: 254705 595 113 282 0 254421 0
[pid=21728] ppid=21716 vsize=300520 CPUtime=0 cores=0-7
/proc/21728/stat : 21728 (python) S 21716 21716 21550 0 -1 4202560 733 0 0 0 0 0 0 0 18 0 1 0 626892119 307732480 5792 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612678 0 0 16781312 514 4252881568596492287 0 0 17 6 0 0 0
/proc/21728/statm: 75130 5792 349 364 0 11042 0
[pid=21730] ppid=21728 vsize=419160 CPUtime=1.11 cores=0-7
/proc/21730/stat : 21730 (adaptg2wsat2009) R 21728 21730 21730 0 -1 4202496 1252 0 0 0 111 0 0 0 25 0 1 0 626892120 429219840 990 33554432000 134512640 135107411 4292946352 18446744073709551615 134523245 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/21730/statm: 104790 990 49 146 0 103666 0
[pid=21731] ppid=21716 vsize=311384 CPUtime=0 cores=0-7
/proc/21731/stat : 21731 (python) S 21716 21716 21550 0 -1 4202560 746 0 0 0 0 0 0 0 18 0 1 0 626892123 318857216 5813 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612678 0 0 16781312 514 4317676338577145855 0 0 17 0 0 0 0
/proc/21731/statm: 77846 5813 349 364 0 13755 0
[pid=21733] ppid=21731 vsize=419160 CPUtime=1.06 cores=0-7
/proc/21733/stat : 21733 (adaptg2wsat2009) R 21731 21733 21733 0 -1 4202496 1254 0 0 0 106 0 0 0 25 0 1 0 626892124 429219840 990 33554432000 134512640 135107411 4287572944 18446744073709551615 134523455 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/21733/statm: 104790 990 49 146 0 103666 0
[pid=21734] ppid=21716 vsize=321572 CPUtime=0 cores=0-7
/proc/21734/stat : 21734 (python) S 21716 21716 21550 0 -1 4202560 757 0 0 0 0 0 0 0 18 0 1 0 626892127 329289728 5817 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612678 0 0 16781312 514 8976190007475699711 0 0 17 4 0 0 0
/proc/21734/statm: 80393 5817 349 364 0 16299 0
[pid=21736] ppid=21734 vsize=419160 CPUtime=1.02 cores=0-7
/proc/21736/stat : 21736 (adaptg2wsat2009) R 21734 21736 21736 0 -1 4202496 1255 0 0 0 102 0 0 0 25 0 1 0 626892127 429219840 992 33554432000 134512640 135107411 4287558592 18446744073709551615 134523294 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/21736/statm: 104790 992 49 146 0 103666 0
[pid=21737] ppid=21716 vsize=331344 CPUtime=0 cores=0-7
/proc/21737/stat : 21737 (python) S 21716 21716 21550 0 -1 4202560 758 0 0 0 0 0 0 0 24 0 1 0 626892131 339296256 5937 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612678 0 0 16781312 514 4264997636978769919 0 0 17 6 0 0 0
/proc/21737/statm: 82836 5937 347 364 0 18739 0
[pid=21739] ppid=21737 vsize=1018820 CPUtime=1 cores=0-7
/proc/21739/stat : 21739 (gnovelty+2) R 21737 21739 21739 0 -1 4202496 860 0 0 0 100 0 0 0 25 0 1 0 626892131 1043271680 595 33554432000 134512640 135663704 4290468384 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/21739/statm: 254705 595 113 282 0 254421 0
[pid=21716/tid=21723] ppid=21714 vsize=362100 CPUtime=0 cores=0-7
/proc/21716/task/21723/stat : 21723 (python) S 21714 21716 21550 0 -1 4202560 26 931 0 0 0 0 0 0 18 0 9 0 626892112 370790400 7492 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 6 0 0 0
[pid=21716/tid=21726] ppid=21714 vsize=362100 CPUtime=0 cores=0-7
/proc/21716/task/21726/stat : 21726 (python) S 21714 21716 21550 0 -1 4202560 24 931 0 0 0 0 0 0 20 0 9 0 626892116 370790400 7492 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
[pid=21716/tid=21729] ppid=21714 vsize=362100 CPUtime=0 cores=0-7
/proc/21716/task/21729/stat : 21729 (python) S 21714 21716 21550 0 -1 4202560 25 931 0 0 0 0 0 0 20 0 9 0 626892120 370790400 7492 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 4 0 0 0
[pid=21716/tid=21732] ppid=21714 vsize=362100 CPUtime=0 cores=0-7
/proc/21716/task/21732/stat : 21732 (python) S 21714 21716 21550 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 9 0 626892123 370790400 7492 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
[pid=21716/tid=21735] ppid=21714 vsize=362100 CPUtime=0 cores=0-7
/proc/21716/task/21735/stat : 21735 (python) S 21714 21716 21550 0 -1 4202560 25 931 0 0 0 0 0 0 22 0 9 0 626892127 370790400 7492 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 4 0 0 0
[pid=21716/tid=21738] ppid=21714 vsize=362100 CPUtime=0 cores=0-7
/proc/21716/task/21738/stat : 21738 (python) S 21714 21716 21550 0 -1 4202560 25 931 0 0 0 0 0 0 24 0 9 0 626892131 370790400 7492 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 8.4
Current children cumulated vsize (KiB) 6511604

[startup+6.30133 s]
/proc/loadavg: 7.15 2.87 1.12 9/286 21745
/proc/meminfo: memFree=23479028/32950940 swapFree=67111364/67111528
[pid=21716] ppid=21714 vsize=362100 CPUtime=1.88 cores=0-7
/proc/21716/stat : 21716 (python) S 21714 21716 21550 0 -1 4202496 18200 931 1 0 176 12 0 0 25 0 9 0 626891921 370790400 7492 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000419275 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/21716/statm: 90525 7492 1891 364 0 26422 0
[pid=21722] ppid=21716 vsize=279940 CPUtime=0 cores=0-7
/proc/21722/stat : 21722 (python) S 21716 21716 21550 0 -1 4202560 896 0 0 0 0 0 0 0 18 0 1 0 626892112 286658560 5796 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612623 0 0 16781312 514 2950478709414396162 0 0 17 5 0 0 0
/proc/21722/statm: 69985 5796 378 364 0 5903 0
[pid=21724] ppid=21722 vsize=1018820 CPUtime=4.38 cores=0-7
/proc/21724/stat : 21724 (gnovelty+2) R 21722 21724 21724 0 -1 4202496 859 0 0 0 438 0 0 0 25 0 1 0 626892112 1043271680 596 33554432000 134512640 135663704 4294328976 18446744073709551615 134525699 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/21724/statm: 254705 596 113 282 0 254421 0
[pid=21725] ppid=21716 vsize=290804 CPUtime=0 cores=0-7
/proc/21725/stat : 21725 (python) S 21716 21716 21550 0 -1 4202560 898 0 0 0 0 0 0 0 18 0 1 0 626892116 297783296 5811 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612678 0 0 16781312 514 4254904120235753730 0 0 17 4 0 0 0
/proc/21725/statm: 72701 5811 372 364 0 8616 0
[pid=21727] ppid=21725 vsize=1018820 CPUtime=4.34 cores=0-7
/proc/21727/stat : 21727 (gnovelty+2) R 21725 21727 21727 0 -1 4202496 858 0 0 0 434 0 0 0 25 0 1 0 626892116 1043271680 595 33554432000 134512640 135663704 4289991568 18446744073709551615 134524888 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/21727/statm: 254705 595 113 282 0 254421 0
[pid=21728] ppid=21716 vsize=300520 CPUtime=0 cores=0-7
/proc/21728/stat : 21728 (python) S 21716 21716 21550 0 -1 4202560 899 0 0 0 0 0 0 0 18 0 1 0 626892119 307732480 5815 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612678 0 0 16781312 514 4252881568596459777 0 0 17 6 0 0 0
/proc/21728/statm: 75130 5815 372 364 0 11042 0
[pid=21730] ppid=21728 vsize=419428 CPUtime=4.3 cores=0-7
/proc/21730/stat : 21730 (adaptg2wsat2009) R 21728 21730 21730 0 -1 4202496 1437 0 0 0 430 0 0 0 25 0 1 0 626892120 429494272 1175 33554432000 134512640 135107411 4292946352 18446744073709551615 134531428 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/21730/statm: 104857 1175 59 146 0 103733 0
[pid=21731] ppid=21716 vsize=311384 CPUtime=0 cores=0-7
/proc/21731/stat : 21731 (python) S 21716 21716 21550 0 -1 4202560 929 0 0 0 0 0 0 0 18 0 1 0 626892123 318857216 5836 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612678 0 0 16781312 514 4317676338577113348 0 0 17 0 0 0 0
/proc/21731/statm: 77846 5836 372 364 0 13755 0
[pid=21733] ppid=21731 vsize=419428 CPUtime=4.23 cores=0-7
/proc/21733/stat : 21733 (adaptg2wsat2009) R 21731 21733 21733 0 -1 4202496 1439 0 0 0 423 0 0 0 25 0 1 0 626892124 429494272 1175 33554432000 134512640 135107411 4287572944 18446744073709551615 134531428 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/21733/statm: 104857 1175 59 146 0 103733 0
[pid=21734] ppid=21716 vsize=321572 CPUtime=0.01 cores=0-7
/proc/21734/stat : 21734 (python) S 21716 21716 21550 0 -1 4202560 935 0 0 0 0 1 0 0 18 0 1 0 626892127 329289728 5840 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612678 0 0 16781312 514 8976190007475667203 0 0 17 4 0 0 0
/proc/21734/statm: 80393 5840 372 364 0 16299 0
[pid=21736] ppid=21734 vsize=419428 CPUtime=4.22 cores=0-7
/proc/21736/stat : 21736 (adaptg2wsat2009) R 21734 21736 21736 0 -1 4202496 1440 0 0 0 422 0 0 0 25 0 1 0 626892127 429494272 1177 33554432000 134512640 135107411 4287558592 18446744073709551615 134531428 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/21736/statm: 104857 1177 59 146 0 103733 0
[pid=21737] ppid=21716 vsize=331344 CPUtime=0 cores=0-7
/proc/21737/stat : 21737 (python) S 21716 21716 21550 0 -1 4202560 943 0 0 0 0 0 0 0 18 0 1 0 626892131 339296256 5960 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612678 0 0 16781312 514 4264997636978737410 0 0 17 6 0 0 0
/proc/21737/statm: 82836 5960 370 364 0 18739 0
[pid=21739] ppid=21737 vsize=1018820 CPUtime=4.19 cores=0-7
/proc/21739/stat : 21739 (gnovelty+2) R 21737 21739 21739 0 -1 4202496 860 0 0 0 419 0 0 0 25 0 1 0 626892131 1043271680 595 33554432000 134512640 135663704 4290468384 18446744073709551615 134525213 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/21739/statm: 254705 595 113 282 0 254421 0
[pid=21740] ppid=21716 vsize=341600 CPUtime=0 cores=0-7
/proc/21740/stat : 21740 (python) S 21716 21716 21550 0 -1 4202560 946 0 0 0 0 0 0 0 18 0 1 0 626892135 349798400 5963 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612678 0 0 16781312 514 8976186159184969987 0 0 17 4 0 0 0
/proc/21740/statm: 85400 5963 370 364 0 21300 0
[pid=21742] ppid=21740 vsize=249192 CPUtime=4.12 cores=0-7
/proc/21742/stat : 21742 (FH) R 21740 21742 21742 0 -1 4202496 1323 0 0 0 412 0 0 0 25 0 1 0 626892135 255172608 1058 33554432000 134512640 134936988 4293556144 18446744073709551615 134553554 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/21742/statm: 62298 1058 41 104 0 61802 0
[pid=21743] ppid=21716 vsize=352448 CPUtime=0 cores=0-7
/proc/21743/stat : 21743 (python) S 21716 21716 21550 0 -1 4202560 980 0 0 0 0 0 0 0 18 0 1 0 626892139 360906752 5976 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612678 0 0 16781312 514 4381593148523020287 0 0 17 6 0 0 0
/proc/21743/statm: 88112 5976 378 364 0 24009 0
[pid=21745] ppid=21743 vsize=419428 CPUtime=4.07 cores=0-7
/proc/21745/stat : 21745 (adaptg2wsat2009) R 21743 21745 21745 0 -1 4202496 1443 0 0 0 407 0 0 0 25 0 1 0 626892139 429494272 1175 33554432000 134512640 135107411 4288664096 18446744073709551615 134532128 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/21745/statm: 104857 1175 59 146 0 103733 0
[pid=21716/tid=21723] ppid=21714 vsize=362100 CPUtime=0 cores=0-7
/proc/21716/task/21723/stat : 21723 (python) S 21714 21716 21550 0 -1 4202560 26 931 0 0 0 0 0 0 18 0 9 0 626892112 370790400 7492 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 6 0 0 0
[pid=21716/tid=21726] ppid=21714 vsize=362100 CPUtime=0 cores=0-7
/proc/21716/task/21726/stat : 21726 (python) S 21714 21716 21550 0 -1 4202560 24 931 0 0 0 0 0 0 20 0 9 0 626892116 370790400 7492 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
[pid=21716/tid=21729] ppid=21714 vsize=362100 CPUtime=0 cores=0-7
/proc/21716/task/21729/stat : 21729 (python) S 21714 21716 21550 0 -1 4202560 25 931 0 0 0 0 0 0 20 0 9 0 626892120 370790400 7492 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 4 0 0 0
[pid=21716/tid=21732] ppid=21714 vsize=362100 CPUtime=0 cores=0-7
/proc/21716/task/21732/stat : 21732 (python) S 21714 21716 21550 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 9 0 626892123 370790400 7492 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
[pid=21716/tid=21735] ppid=21714 vsize=362100 CPUtime=0 cores=0-7
/proc/21716/task/21735/stat : 21735 (python) S 21714 21716 21550 0 -1 4202560 25 931 0 0 0 0 0 0 22 0 9 0 626892127 370790400 7492 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 4 0 0 0
[pid=21716/tid=21738] ppid=21714 vsize=362100 CPUtime=0 cores=0-7
/proc/21716/task/21738/stat : 21738 (python) S 21714 21716 21550 0 -1 4202560 25 931 0 0 0 0 0 0 24 0 9 0 626892131 370790400 7492 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
[pid=21716/tid=21741] ppid=21714 vsize=362100 CPUtime=0 cores=0-7
/proc/21716/task/21741/stat : 21741 (python) S 21714 21716 21550 0 -1 4202560 26 931 0 0 0 0 0 0 25 0 9 0 626892135 370790400 7492 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 4 0 0 0
[pid=21716/tid=21744] ppid=21714 vsize=362100 CPUtime=0 cores=0-7
/proc/21716/task/21744/stat : 21744 (python) S 21714 21716 21550 0 -1 4202560 84 931 0 0 0 0 0 0 25 0 9 0 626892139 370790400 7492 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 6 0 0 0

################
# More data... #
################

/proc/21812/task/21821/stat : 21821 (python) S 21716 21716 21550 0 -1 4202560 99 0 0 0 0 0 0 0 19 0 2 0 626941867 574943232 6466 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 514 18446604469177853120 0 0 -1 0 0 0 0
[pid=21818] ppid=21716 vsize=571724 CPUtime=0.21 cores=0-7
/proc/21818/stat : 21818 (python) S 21716 21716 21550 0 -1 4202560 1197 0 0 0 10 11 0 0 19 0 2 0 626941863 585445376 6474 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000419275 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/21818/statm: 142931 6474 407 364 0 78783 0
[pid=21820] ppid=21818 vsize=335996 CPUtime=99.94 cores=0-7
/proc/21820/stat : 21820 (iPAWS) T 21818 21820 21820 0 -1 4202496 1707 0 0 0 9994 0 0 0 25 0 1 0 626941863 344059904 1428 33554432000 134512640 135114423 4288876224 18446744073709551615 134516669 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/21820/statm: 83999 1428 59 147 0 83851 0
[pid=21818/tid=21833] ppid=21716 vsize=571724 CPUtime=0 cores=0-7
/proc/21818/task/21833/stat : 21833 (python) S 21716 21716 21550 0 -1 4202560 106 0 0 0 0 0 0 0 19 0 2 0 626951884 585445376 6474 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 514 18446604469177854016 0 0 -1 1 0 0 0
[pid=21822] ppid=21716 vsize=581980 CPUtime=0.42 cores=0-7
/proc/21822/stat : 21822 (python) S 21716 21716 21550 0 -1 4202560 1206 0 0 0 22 20 0 0 19 0 2 0 626941884 595947520 6603 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000419275 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/21822/statm: 145495 6603 406 364 0 81344 0
[pid=21824] ppid=21822 vsize=268420 CPUtime=196.62 cores=0-7
/proc/21824/stat : 21824 (hybridGM3) T 21822 21824 21824 0 -1 4202496 5292 0 0 0 19662 0 0 0 25 0 1 0 626941885 274862080 5008 33554432000 134512640 135204385 4288438032 18446744073709551615 134537939 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/21824/statm: 67105 5008 63 169 0 66935 0
[pid=21822/tid=21842] ppid=21716 vsize=581980 CPUtime=0 cores=0-7
/proc/21822/task/21842/stat : 21842 (python) S 21716 21716 21550 0 -1 4202560 97 0 0 0 0 0 0 0 19 0 2 0 626961555 595947520 6603 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 514 18446604453908981888 0 0 -1 0 0 0 0
[pid=21830] ppid=21716 vsize=592816 CPUtime=0.91 cores=0-7
/proc/21830/stat : 21830 (python) S 21716 21716 21550 0 -1 4202560 1270 0 0 0 51 40 0 0 18 0 2 0 626951605 607043584 6464 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000419275 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/21830/statm: 148204 6464 401 364 0 84050 0
[pid=21832] ppid=21830 vsize=268424 CPUtime=390.98 cores=0-7
/proc/21832/stat : 21832 (hybridGM3) T 21830 21832 21832 0 -1 4202496 5300 0 0 0 39097 1 0 0 25 0 1 0 626951606 274866176 5010 33554432000 134512640 135204385 4292510608 18446744073709551615 134545423 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/21832/statm: 67106 5010 63 169 0 66936 0
[pid=21830/tid=21853] ppid=21716 vsize=592816 CPUtime=0 cores=0-7
/proc/21830/task/21853/stat : 21853 (python) S 21716 21716 21550 0 -1 4202560 45 0 0 0 0 0 0 0 18 0 2 0 626971157 607043584 6464 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 514 18446604453713558336 0 0 -1 2 0 0 0
[pid=21834] ppid=21716 vsize=602660 CPUtime=0.22 cores=0-7
/proc/21834/stat : 21834 (python) S 21716 21716 21550 0 -1 4202560 1186 0 0 0 13 9 0 0 19 0 2 0 626952323 617123840 6619 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000419275 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/21834/statm: 150665 6619 401 364 0 86508 0
[pid=21836] ppid=21834 vsize=335996 CPUtime=98.64 cores=0-7
/proc/21836/stat : 21836 (iPAWS) T 21834 21836 21836 0 -1 4202496 1706 0 0 0 9862 2 0 0 25 0 1 0 626952323 344059904 1428 33554432000 134512640 135114423 4291376304 18446744073709551615 134516632 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/21836/statm: 83999 1428 59 147 0 83851 0
[pid=21834/tid=21846] ppid=21716 vsize=602660 CPUtime=0 cores=0-7
/proc/21834/task/21846/stat : 21846 (python) S 21716 21716 21550 0 -1 4202560 102 0 0 0 0 0 0 0 19 0 2 0 626962249 617123840 6619 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 514 18446604449149720576 0 0 -1 5 0 0 0
[pid=21839] ppid=21716 vsize=612916 CPUtime=0.45 cores=0-7
/proc/21839/stat : 21839 (python) S 21716 21716 21550 0 -1 4202560 1188 0 0 0 24 21 0 0 19 0 2 0 626960945 627625984 6674 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000419275 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/21839/statm: 153229 6674 400 364 0 89069 0
[pid=21841] ppid=21839 vsize=268424 CPUtime=195.96 cores=0-7
/proc/21841/stat : 21841 (hybridGM3) T 21839 21841 21841 0 -1 4202496 5292 0 0 0 19596 0 0 0 25 0 1 0 626960946 274866176 5009 33554432000 134512640 135204385 4289613424 18446744073709551615 134537792 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/21841/statm: 67106 5009 63 169 0 66936 0
[pid=21839/tid=21856] ppid=21716 vsize=612916 CPUtime=0 cores=0-7
/proc/21839/task/21856/stat : 21856 (python) S 21716 21716 21550 0 -1 4202560 98 0 0 0 0 0 0 0 19 0 2 0 626980595 627625984 6674 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 514 18446604451984274688 0 0 -1 1 0 0 0
[pid=21843] ppid=21716 vsize=623172 CPUtime=0.7 cores=0-7
/proc/21843/stat : 21843 (python) S 21716 21716 21550 0 -1 4202560 1209 0 0 0 39 31 0 0 18 0 2 0 626961565 638128128 6734 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000419275 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/21843/statm: 155793 6734 401 364 0 91630 0
[pid=21845] ppid=21843 vsize=268420 CPUtime=295.63 cores=0-7
/proc/21845/stat : 21845 (hybridGM3) T 21843 21845 21845 0 -1 4202496 5290 0 0 0 29562 1 0 0 25 0 1 0 626961565 274862080 5008 33554432000 134512640 135204385 4293140528 18446744073709551615 134537939 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/21845/statm: 67105 5008 63 169 0 66935 0
[pid=21843/tid=21860] ppid=21716 vsize=623172 CPUtime=0 cores=0-7
/proc/21843/task/21860/stat : 21860 (python) S 21716 21716 21550 0 -1 4202560 100 0 0 0 0 0 0 0 18 0 2 0 626981214 638128128 6734 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 514 18446604453985825216 0 0 -1 4 0 0 0
[pid=21847] ppid=21716 vsize=633428 CPUtime=0.9 cores=0-7
/proc/21847/stat : 21847 (python) S 21716 21716 21550 0 -1 4202560 1031 0 0 0 48 42 0 0 18 0 1 0 626962259 648630272 6654 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612678 0 0 16781312 514 16440038595995074819 0 0 17 0 0 0 0
/proc/21847/statm: 158357 6654 378 364 0 94191 0
[pid=21849] ppid=21847 vsize=457664 CPUtime=389.83 cores=0-7
/proc/21849/stat : 21849 (TNM) R 21847 21849 21849 0 -1 4202496 1326 0 0 0 38983 0 0 0 25 0 1 0 626962260 468647936 1037 33554432000 134512640 135008748 4289700352 18446744073709551615 134529446 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/21849/statm: 114416 1037 46 122 0 113316 0
[pid=21857] ppid=21716 vsize=643684 CPUtime=0.48 cores=0-7
/proc/21857/stat : 21857 (python) S 21716 21716 21550 0 -1 4202560 1255 0 0 0 23 25 0 0 18 0 2 0 626980605 659132416 6631 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612678 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/21857/statm: 160921 6631 407 364 0 96752 0
[pid=21859] ppid=21857 vsize=249188 CPUtime=204.56 cores=0-7
/proc/21859/stat : 21859 (FH) R 21857 21859 21859 0 -1 4202496 1343 0 0 0 20455 1 0 0 25 0 1 0 626980607 255168512 1058 33554432000 134512640 134936988 4294853568 18446744073709551615 134553279 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/21859/statm: 62297 1058 41 104 0 61801 0
[pid=21857/tid=21864] ppid=21716 vsize=643684 CPUtime=0 cores=0-7
/proc/21857/task/21864/stat : 21864 (python) S 21716 21716 21550 0 -1 4202560 108 0 0 0 0 0 0 0 19 0 2 0 626990629 659132416 6631 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 514 18446604453906306176 0 0 -1 0 0 0 0
[pid=21865] ppid=21716 vsize=653940 CPUtime=0.25 cores=0-7
/proc/21865/stat : 21865 (python) S 21716 21716 21550 0 -1 4202560 1039 0 0 0 13 12 0 0 18 0 1 0 626990639 669634560 6612 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612678 0 0 16781312 514 16440038595995074819 0 0 17 1 0 0 0
/proc/21865/statm: 163485 6612 379 364 0 99313 0
[pid=21867] ppid=21865 vsize=419076 CPUtime=106.96 cores=0-7
/proc/21867/stat : 21867 (NCVWr) R 21865 21867 21867 0 -1 4202496 1362 0 0 0 10696 0 0 0 25 0 1 0 626990640 429133824 1078 33554432000 134512640 135113558 4292164192 18446744073709551615 134533476 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/21867/statm: 104769 1078 58 147 0 103644 0
[pid=21868] ppid=21716 vsize=664196 CPUtime=0.23 cores=0-7
/proc/21868/stat : 21868 (python) S 21716 21716 21550 0 -1 4202560 1252 0 0 0 12 11 0 0 19 0 2 0 626991263 680136704 6655 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000419275 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/21868/statm: 166049 6655 406 364 0 101874 0
[pid=21870] ppid=21868 vsize=1018820 CPUtime=99.79 cores=0-7
/proc/21870/stat : 21870 (gnovelty+2) T 21868 21870 21870 0 -1 4202496 883 0 0 0 9979 0 0 0 25 0 1 0 626991263 1043271680 596 33554432000 134512640 135663704 4287857952 18446744073709551615 134525255 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/21870/statm: 254705 596 113 282 0 254421 0
[pid=21868/tid=21874] ppid=21716 vsize=664196 CPUtime=0 cores=0-7
/proc/21868/task/21874/stat : 21874 (python) S 21716 21716 21550 0 -1 4202560 100 0 0 0 0 0 0 0 19 0 2 0 627001289 680136704 6655 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 514 18446604453921115072 0 0 -1 1 0 0 0
[pid=21875] ppid=21716 vsize=674452 CPUtime=0 cores=0-7
/proc/21875/stat : 21875 (python) S 21716 21716 21550 0 -1 4202560 937 0 0 0 0 0 0 0 19 0 1 0 627001300 690638848 6573 33554432000 4194304 5683644 140734622909168 18446744073709551615 215988612678 0 0 16781312 514 4369935301611618303 0 0 17 0 0 0 0
/proc/21875/statm: 168613 6573 349 364 0 104435 0
[pid=21877] ppid=21875 vsize=268424 CPUtime=0.62 cores=0-7
/proc/21877/stat : 21877 (hybridGM3) R 21875 21877 21877 0 -1 4202496 5299 0 0 0 62 0 0 0 22 0 1 0 627001301 274866176 5008 33554432000 134512640 135204385 4294691920 18446744073709551615 134538427 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/21877/statm: 67106 5008 63 169 0 66936 0
[pid=21716/tid=21723] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21723/stat : 21723 (python) S 21714 21716 21550 0 -1 4202560 26 931 0 0 0 0 0 0 18 0 34 0 626892112 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 6 0 0 0
[pid=21716/tid=21726] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21726/stat : 21726 (python) S 21714 21716 21550 0 -1 4202560 24 931 0 0 0 0 0 0 20 0 34 0 626892116 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
[pid=21716/tid=21729] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21729/stat : 21729 (python) S 21714 21716 21550 0 -1 4202560 39 931 0 0 0 0 0 0 18 0 34 0 626892120 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 4 0 0 0
[pid=21716/tid=21732] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21732/stat : 21732 (python) S 21714 21716 21550 0 -1 4202560 31 931 0 0 0 0 0 0 18 0 34 0 626892123 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
[pid=21716/tid=21735] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21735/stat : 21735 (python) S 21714 21716 21550 0 -1 4202560 29 931 0 0 0 0 0 0 18 0 34 0 626892127 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 4 0 0 0
[pid=21716/tid=21738] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21738/stat : 21738 (python) S 21714 21716 21550 0 -1 4202560 25 931 0 0 0 0 0 0 24 0 34 0 626892131 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
[pid=21716/tid=21741] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21741/stat : 21741 (python) S 21714 21716 21550 0 -1 4202560 31 931 0 0 0 0 0 0 18 0 34 0 626892135 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 4 0 0 0
[pid=21716/tid=21744] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21744/stat : 21744 (python) S 21714 21716 21550 0 -1 4202560 88 931 0 0 0 0 0 0 18 0 34 0 626892139 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 6 0 0 0
[pid=21716/tid=21751] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21751/stat : 21751 (python) S 21714 21716 21550 0 -1 4202560 35 931 0 0 0 0 0 0 18 0 34 0 626902139 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 1 0 0 0
[pid=21716/tid=21756] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21756/stat : 21756 (python) S 21714 21716 21550 0 -1 4202560 115 931 0 0 0 0 0 0 18 0 34 0 626902145 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 5 0 0 0
[pid=21716/tid=21760] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21760/stat : 21760 (python) S 21714 21716 21550 0 -1 4202560 44 931 0 0 0 0 0 0 18 0 34 0 626902149 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 7 0 0 0
[pid=21716/tid=21764] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21764/stat : 21764 (python) S 21714 21716 21550 0 -1 4202560 30 931 0 0 0 0 0 0 25 0 34 0 626902155 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 1 0 0 0
[pid=21716/tid=21769] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21769/stat : 21769 (python) S 21714 21716 21550 0 -1 4202560 27 931 0 0 0 0 0 0 18 0 34 0 626902160 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 7 0 0 0
[pid=21716/tid=21772] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21772/stat : 21772 (python) S 21714 21716 21550 0 -1 4202560 32 931 0 0 0 0 0 0 18 0 34 0 626902166 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 7 0 0 0
[pid=21716/tid=21775] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21775/stat : 21775 (python) S 21714 21716 21550 0 -1 4202560 24 931 0 0 0 0 0 0 18 0 34 0 626902172 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 1 0 0 0
[pid=21716/tid=21778] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21778/stat : 21778 (python) S 21714 21716 21550 0 -1 4202560 29 931 0 0 0 0 0 0 18 0 34 0 626902179 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 7 0 0 0
[pid=21716/tid=21786] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21786/stat : 21786 (python) S 21714 21716 21550 0 -1 4202560 22 931 0 0 0 0 0 0 19 0 34 0 626912169 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 7 0 0 0
[pid=21716/tid=21789] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21789/stat : 21789 (python) S 21714 21716 21550 0 -1 4202560 35 931 0 0 0 0 0 0 18 0 34 0 626912176 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 7 0 0 0
[pid=21716/tid=21801] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21801/stat : 21801 (python) S 21714 21716 21550 0 -1 4202560 35 931 0 0 0 0 0 0 18 0 34 0 626922535 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 2 0 0 0
[pid=21716/tid=21806] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21806/stat : 21806 (python) S 21714 21716 21550 0 -1 4202560 146 931 0 0 0 0 0 0 18 0 34 0 626931353 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
[pid=21716/tid=21809] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21809/stat : 21809 (python) S 21714 21716 21550 0 -1 4202560 38 931 0 0 0 0 0 0 20 0 34 0 626931641 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 1 0 0 0
[pid=21716/tid=21813] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21813/stat : 21813 (python) S 21714 21716 21550 0 -1 4202560 35 931 0 0 0 0 0 0 20 0 34 0 626931837 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 1 0 0 0
[pid=21716/tid=21819] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21819/stat : 21819 (python) S 21714 21716 21550 0 -1 4202560 48 931 0 0 0 0 0 0 20 0 34 0 626941863 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
[pid=21716/tid=21823] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21823/stat : 21823 (python) S 21714 21716 21550 0 -1 4202560 39 931 0 0 0 0 0 0 21 0 34 0 626941884 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
[pid=21716/tid=21831] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21831/stat : 21831 (python) S 21714 21716 21550 0 -1 4202560 42 931 0 0 0 0 0 0 18 0 34 0 626951606 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 1 0 0 0
[pid=21716/tid=21835] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21835/stat : 21835 (python) S 21714 21716 21550 0 -1 4202560 42 931 0 0 0 0 0 0 20 0 34 0 626952323 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
[pid=21716/tid=21840] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21840/stat : 21840 (python) S 21714 21716 21550 0 -1 4202560 35 931 0 0 0 0 0 0 19 0 34 0 626960945 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
[pid=21716/tid=21844] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21844/stat : 21844 (python) S 21714 21716 21550 0 -1 4202560 39 931 0 0 0 0 0 0 18 0 34 0 626961565 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
[pid=21716/tid=21848] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21848/stat : 21848 (python) S 21714 21716 21550 0 -1 4202560 36 931 0 0 0 0 0 0 20 0 34 0 626962260 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
[pid=21716/tid=21858] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21858/stat : 21858 (python) S 21714 21716 21550 0 -1 4202560 49 931 0 0 0 0 0 0 18 0 34 0 626980605 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
[pid=21716/tid=21866] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21866/stat : 21866 (python) S 21714 21716 21550 0 -1 4202560 42 931 0 0 0 0 0 0 19 0 34 0 626990640 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 0 0 0 0
[pid=21716/tid=21869] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21869/stat : 21869 (python) S 21714 21716 21550 0 -1 4202560 38 931 0 0 0 0 0 0 20 0 34 0 626991263 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 1 0 0 0
[pid=21716/tid=21876] ppid=21714 vsize=684696 CPUtime=0 cores=0-7
/proc/21716/task/21876/stat : 21876 (python) S 21714 21716 21550 0 -1 4202560 42 931 0 0 0 0 0 0 20 0 34 0 627001300 701128704 8170 33554432000 4194304 5683644 140734622909168 18446744073709551615 216000416177 0 0 16781312 2 18446604453985827904 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 8703.62
Current children cumulated vsize (KiB) 32114608

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 21716 and gives
#  childrusage.ru_utime.tv_sec=2097
#  childrusage.ru_utime.tv_usec=669105
#  childrusage.ru_stime.tv_sec=7
#  childrusage.ru_stime.tv_usec=547852
# CPU time returned by wait4() is 2105.22
# while last known CPU time is 8703.62
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1094.61
CPU time (s): 8703.62
CPU user time (s): 8694.08
CPU system time (s): 9.54
CPU usage (%): 795.135
Max. virtual memory (cumulated for all children) (KiB): 32114608

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2097.67
system time used= 7.54785
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 96695
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 12982
involuntary context switches= 7265

runsolver used 10.7374 second user time and 24.9852 second system time

The end

Launcher Data 

Begin job on node117 at 2011-05-02 22:03:31
IDJOB=3321752
IDBENCH=82530
IDSOLVER=1712
FILE ID=node117/3321752-1304366611
RUNJOBID= node117-1304366467-21564
PBS_JOBID= 13181754
Free space on /tmp= 72672 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k7-r85-v250-c21250-S381985918-018-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321752-1304366611/watcher-3321752-1304366611 -o /tmp/evaluation-result-3321752-1304366611/solver-3321752-1304366611 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321752-1304366611.cnf -seed 353230720 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 13d193ddef22075908d36a7c0d4a2be1
RANDOM SEED=353230720

node117.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5267.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950940 kB
MemFree:      23562772 kB
Buffers:       1021756 kB
Cached:        4502288 kB
SwapCached:        160 kB
Active:        3818852 kB
Inactive:      5077808 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950940 kB
LowFree:      23562772 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:           32224 kB
Writeback:           0 kB
AnonPages:     3372608 kB
Mapped:          23492 kB
Slab:           408440 kB
PageTables:      19628 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586996 kB
Committed_AS: 37496616 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72640 MiB
End job on node117 at 2011-05-02 22:21:48