Trace number 3281488

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Satisfiability Solver Selector 2011-03-18? (TO) 1300.1 1300.01

General information on the benchmark

NameSAT_RACE08/Q1/
SAT_dat.k85.cnf
MD5SUMdce41996d9c5153aa0c253fa128d227c
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark43.2814
Satisfiable
(Un)Satisfiability was proved
Number of variables181484
Number of clauses890298
Sum of the clauses size2349692
Maximum clause length19
Minimum clause length1
Number of clauses of size 1771
Number of clauses of size 2667172
Number of clauses of size 3144951
Number of clauses of size 424791
Number of clauses of size 510950
Number of clauses of size over 541663

Solver Data 

1300.10/1300.01	c COMMAND LINE: portfolio_solver.py -t 2400 -s 855434744 HOME/instance-3281488-1303751363.cnf
1300.10/1300.01	c    tmpdir         WORKING-DIR
1300.10/1300.01	c    timelimit      2400
1300.10/1300.01	c    seed           855434744
1300.10/1300.01	c    verbosity      0
1300.10/1300.01	c Parsed command line and set default values...
1300.10/1300.01	c Starting *WALL CLOCK* TIMER
1300.10/1300.01	c Found WORKING-DIR/coach-executables-test-SUCCESS.txt
1300.10/1300.01	c   indicating coach-executables-test.sh was run successfully earlier
1300.10/1300.01	c Making sure all necessary files and folders are present...
1300.10/1300.01	c   Checking for ./features                            OK
1300.10/1300.01	c   Checking for COACH/coach                           OK
1300.10/1300.01	c   Checking for preprocessors/SatELite_v1.0_linux     OK
1300.10/1300.01	c   Checking for solvers/adaptg2wsat2009               OK
1300.10/1300.01	c   Checking for solvers/adaptg2wsat2009++             OK
1300.10/1300.01	c   Checking for solvers/clasp-1.3.6-x86-linux         OK
1300.10/1300.01	c   Checking for solvers/cryptominisat                 OK
1300.10/1300.01	c   Checking for solvers/glucose_static                OK
1300.10/1300.01	c   Checking for solvers/gnovelty+2                    OK
1300.10/1300.01	c   Checking for solvers/gnovelty+2-H                  OK
1300.10/1300.01	c   Checking for solvers/hybridGM3                     OK
1300.10/1300.01	c   Checking for solvers/kcnfs04SAT07                  OK
1300.10/1300.01	c   Checking for solvers/lingeling                     OK
1300.10/1300.01	c   Checking for solvers/LySATc                        OK
1300.10/1300.01	c   Checking for solvers/LySATi                        OK
1300.10/1300.01	c   Checking for solvers/march_hi                      OK
1300.10/1300.01	c   Checking for solvers/march_nn                      OK
1300.10/1300.01	c   Checking for solvers/minisat_static                OK
1300.10/1300.01	c   Checking for solvers/mxc-sat09                     OK
1300.10/1300.01	c   Checking for solvers/picosat                       OK
1300.10/1300.01	c   Checking for solvers/precosat                      OK
1300.10/1300.01	c   Checking for solvers/SATenstein                    OK
1300.10/1300.01	c   Checking for solvers/TNM                           OK
1300.10/1300.01	c   Checking for solvers/ubcsatOrig                    OK
1300.10/1300.01	c   Checking for utilities/timerun                     OK
1300.10/1300.01	c   Checking for folder COACH/DATA                     OK
1300.10/1300.01	c   Checking for file unif-k3-r4.2-v18000-c75600-S914323646-060.cnf OK
1300.10/1300.01	c Check successful
1300.10/1300.01	c TIMER: 0.0 sec total, 0.0 sec since previous report
1300.10/1300.01	c Computing instance features...
1300.10/1300.01	c    ./features HOME/instance-3281488-1303751363.cnf
1300.10/1300.01	c ERROR: 48 features not computed correctly!
1300.10/1300.01	c Calling default solver with no time limit!
1300.10/1300.01	c    solvers/lingeling HOME/instance-3281488-1303751363.cnf
1300.10/1300.01	c TIMER: 1299.98 sec total, 1299.98 sec since previous report
1300.10/1300.01	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3281488-1303751363/watcher-3281488-1303751363 -o /tmp/evaluation-result-3281488-1303751363/solver-3281488-1303751363 -C 2400 -W 1300 -M 7680 python portfolio_solver.py -t 2400 -s 855434744 HOME/instance-3281488-1303751363.cnf 

running on 2 cores: 5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.00 1.74 2.70 4/184 19422
/proc/meminfo: memFree=16513988/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=15444 CPUtime=0 cores=5,7
/proc/19422/stat : 19422 (python) R 19420 19422 18696 0 -1 4202496 816 0 0 0 0 0 0 0 17 0 1 0 486736810 15814656 705 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/19422/statm: 3861 706 344 379 0 334 0

[startup+0.0243389 s]
/proc/loadavg: 1.00 1.74 2.70 4/184 19422
/proc/meminfo: memFree=16513988/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=16852 CPUtime=0.01 cores=5,7
/proc/19422/stat : 19422 (python) R 19420 19422 18696 0 -1 4202496 1234 0 0 0 1 0 0 0 17 0 1 0 486736810 17256448 1102 33554432000 4194304 5745468 140736862150912 18446744073709551615 4547741 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/19422/statm: 4213 1105 435 379 0 686 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16852

[startup+0.100334 s]
/proc/loadavg: 1.00 1.74 2.70 4/184 19422
/proc/meminfo: memFree=16513988/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=21952 CPUtime=0.01 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 1527 0 0 0 1 0 0 0 17 0 1 0 486736810 22478848 1366 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 5 0 0 0
/proc/19422/statm: 5488 1366 463 379 0 929 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 21952

[startup+0.300288 s]
/proc/loadavg: 1.00 1.74 2.70 4/184 19422
/proc/meminfo: memFree=16513988/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=21952 CPUtime=0.01 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 1527 0 0 0 1 0 0 0 17 0 1 0 486736810 22478848 1366 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 5 0 0 0
/proc/19422/statm: 5488 1366 463 379 0 929 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 21952

[startup+0.700212 s]
/proc/loadavg: 1.00 1.74 2.70 4/184 19422
/proc/meminfo: memFree=16513988/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=21952 CPUtime=0.01 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 1527 0 0 0 1 0 0 0 17 0 1 0 486736810 22478848 1366 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 5 0 0 0
/proc/19422/statm: 5488 1366 463 379 0 929 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 21952

[startup+1.50107 s]
/proc/loadavg: 1.24 1.78 2.71 5/190 19431
/proc/meminfo: memFree=16411692/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=21952 CPUtime=0.01 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 1527 0 0 0 1 0 0 0 17 0 1 0 486736810 22478848 1366 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 5 0 0 0
/proc/19422/statm: 5488 1366 463 379 0 929 0
[pid=19424] ppid=19422 vsize=8364 CPUtime=1.46 cores=5,7
/proc/19424/stat : 19424 (features) R 19422 19422 18696 0 -1 4202496 2640 0 0 0 144 2 0 0 21 0 1 0 486736813 8564736 1523 33554432000 134512640 135907751 4288880624 18446744073709551615 135392733 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/19424/statm: 2091 1523 111 341 0 1747 0
Current children cumulated CPU time (s) 1.47
Current children cumulated vsize (KiB) 30316

[startup+3.10074 s]
/proc/loadavg: 1.24 1.78 2.71 5/194 19438
/proc/meminfo: memFree=16363284/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
Current children cumulated CPU time (s) 2.53
Current children cumulated vsize (KiB) 57792

[startup+6.30118 s]
/proc/loadavg: 1.24 1.78 2.71 5/196 19440
/proc/meminfo: memFree=16101756/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
[pid=19440] ppid=19422 vsize=38284 CPUtime=3.82 cores=5,7
/proc/19440/stat : 19440 (lingeling) R 19422 19422 18696 0 -1 4202496 26512 0 0 0 378 4 0 0 25 0 1 0 486737057 39202816 9288 33554432000 134512640 134650204 4294757856 18446744073709551615 134542562 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/19440/statm: 9571 9288 111 34 0 9186 0
Current children cumulated CPU time (s) 6.35
Current children cumulated vsize (KiB) 96076

[startup+12.701 s]
/proc/loadavg: 1.89 1.90 2.74 5/196 19441
/proc/meminfo: memFree=15907100/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
[pid=19440] ppid=19422 vsize=61764 CPUtime=10.22 cores=5,7
/proc/19440/stat : 19440 (lingeling) R 19422 19422 18696 0 -1 4202496 32382 0 0 0 1017 5 0 0 25 0 1 0 486737057 63246336 15158 33554432000 134512640 134650204 4294757856 18446744073709551615 134528138 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/19440/statm: 15441 15158 111 34 0 15056 0
Current children cumulated CPU time (s) 12.75
Current children cumulated vsize (KiB) 119556

[startup+25.5006 s]
/proc/loadavg: 2.21 1.97 2.75 5/196 19441
/proc/meminfo: memFree=15917176/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
[pid=19440] ppid=19422 vsize=55592 CPUtime=23.01 cores=5,7
/proc/19440/stat : 19440 (lingeling) R 19422 19422 18696 0 -1 4202496 47851 0 0 0 2294 7 0 0 25 0 1 0 486737057 56926208 13584 33554432000 134512640 134650204 4294757856 18446744073709551615 134585457 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/19440/statm: 13898 13584 112 34 0 13513 0
Current children cumulated CPU time (s) 25.54
Current children cumulated vsize (KiB) 113384

[startup+51.1009 s]
/proc/loadavg: 2.97 2.18 2.80 5/196 19442
/proc/meminfo: memFree=15895664/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
[pid=19440] ppid=19422 vsize=57160 CPUtime=48.62 cores=5,7
/proc/19440/stat : 19440 (lingeling) R 19422 19422 18696 0 -1 4202496 72483 0 0 0 4851 11 0 0 25 0 1 0 486737057 58531840 13976 33554432000 134512640 134650204 4294757856 18446744073709551615 134630310 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/19440/statm: 14290 13976 112 34 0 13905 0
Current children cumulated CPU time (s) 51.15
Current children cumulated vsize (KiB) 114952

[startup+102.3 s]
/proc/loadavg: 3.61 2.47 2.86 5/201 19477
/proc/meminfo: memFree=15522436/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
[pid=19440] ppid=19422 vsize=58908 CPUtime=99.82 cores=5,7
/proc/19440/stat : 19440 (lingeling) R 19422 19422 18696 0 -1 4202496 101245 0 0 0 9967 15 0 0 25 0 1 0 486737057 60321792 14444 33554432000 134512640 134650204 4294757856 18446744073709551615 134631426 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/19440/statm: 14727 14444 112 34 0 14342 0
Current children cumulated CPU time (s) 102.35
Current children cumulated vsize (KiB) 116700

[startup+162.301 s]
/proc/loadavg: 3.85 2.75 2.93 6/204 19493
/proc/meminfo: memFree=15827712/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
[pid=19440] ppid=19422 vsize=58748 CPUtime=159.83 cores=5,7
/proc/19440/stat : 19440 (lingeling) R 19422 19422 18696 0 -1 4202496 129684 0 0 0 15964 19 0 0 25 0 1 0 486737057 60157952 14405 33554432000 134512640 134650204 4294757856 18446744073709551615 134553093 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/19440/statm: 14687 14405 112 34 0 14302 0
Current children cumulated CPU time (s) 162.36
Current children cumulated vsize (KiB) 116540

[startup+222.301 s]
/proc/loadavg: 3.94 2.97 3.00 6/204 19495
/proc/meminfo: memFree=15802636/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
[pid=19440] ppid=19422 vsize=59496 CPUtime=219.83 cores=5,7
/proc/19440/stat : 19440 (lingeling) R 19422 19422 18696 0 -1 4202496 150438 0 0 0 21961 22 0 0 25 0 1 0 486737057 60923904 14567 33554432000 134512640 134650204 4294757856 18446744073709551615 134568248 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/19440/statm: 14874 14567 112 34 0 14489 0
Current children cumulated CPU time (s) 222.36
Current children cumulated vsize (KiB) 117288

[startup+282.301 s]
/proc/loadavg: 3.98 3.16 3.05 6/204 19503
/proc/meminfo: memFree=15472452/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7

################
# More data... #
################

/proc/meminfo: memFree=12943816/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
[pid=19440] ppid=19422 vsize=70260 CPUtime=759.85 cores=5,7
/proc/19440/stat : 19440 (lingeling) R 19422 19422 18696 0 -1 4202496 284734 0 0 0 75946 39 0 0 25 0 1 0 486737057 71946240 17283 33554432000 134512640 134650204 4294757856 18446744073709551615 134546416 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/19440/statm: 17565 17283 112 34 0 17180 0
Current children cumulated CPU time (s) 762.38
Current children cumulated vsize (KiB) 128052

[startup+822.301 s]
/proc/loadavg: 4.06 3.88 3.46 5/201 19587
/proc/meminfo: memFree=12942400/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
[pid=19440] ppid=19422 vsize=65112 CPUtime=819.84 cores=5,7
/proc/19440/stat : 19440 (lingeling) R 19422 19422 18696 0 -1 4202496 297016 0 0 0 81944 40 0 0 25 0 1 0 486737057 66674688 15996 33554432000 134512640 134650204 4294757856 18446744073709551615 134630278 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/19440/statm: 16278 15996 112 34 0 15893 0
Current children cumulated CPU time (s) 822.37
Current children cumulated vsize (KiB) 122904

[startup+882.3 s]
/proc/loadavg: 4.08 3.91 3.50 5/201 19589
/proc/meminfo: memFree=12666184/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
[pid=19440] ppid=19422 vsize=72336 CPUtime=879.85 cores=5,7
/proc/19440/stat : 19440 (lingeling) R 19422 19422 18696 0 -1 4202496 311517 0 0 0 87943 42 0 0 25 0 1 0 486737057 74072064 17802 33554432000 134512640 134650204 4294757856 18446744073709551615 134630242 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/19440/statm: 18084 17802 112 34 0 17699 0
Current children cumulated CPU time (s) 882.38
Current children cumulated vsize (KiB) 130128

[startup+942.3 s]
/proc/loadavg: 4.03 3.93 3.53 5/204 19597
/proc/meminfo: memFree=12570968/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
[pid=19440] ppid=19422 vsize=79208 CPUtime=939.85 cores=5,7
/proc/19440/stat : 19440 (lingeling) R 19422 19422 18696 0 -1 4202496 319398 0 0 0 93942 43 0 0 25 0 1 0 486737057 81108992 19520 33554432000 134512640 134650204 4294757856 18446744073709551615 134547823 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/19440/statm: 19802 19520 112 34 0 19417 0
Current children cumulated CPU time (s) 942.38
Current children cumulated vsize (KiB) 137000

[startup+1002.3 s]
/proc/loadavg: 4.18 3.98 3.57 5/201 19627
/proc/meminfo: memFree=12443604/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
[pid=19440] ppid=19422 vsize=67484 CPUtime=999.85 cores=5,7
/proc/19440/stat : 19440 (lingeling) R 19422 19422 18696 0 -1 4202496 331297 0 0 0 99941 44 0 0 25 0 1 0 486737057 69103616 16589 33554432000 134512640 134650204 4294757856 18446744073709551615 134543832 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/19440/statm: 16871 16589 112 34 0 16486 0
Current children cumulated CPU time (s) 1002.38
Current children cumulated vsize (KiB) 125276

[startup+1062.3 s]
/proc/loadavg: 4.06 3.98 3.59 5/204 19631
/proc/meminfo: memFree=12200756/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
[pid=19440] ppid=19422 vsize=73804 CPUtime=1059.86 cores=5,7
/proc/19440/stat : 19440 (lingeling) R 19422 19422 18696 0 -1 4202496 341331 0 0 0 105940 46 0 0 25 0 1 0 486737057 75575296 18141 33554432000 134512640 134650204 4294757856 18446744073709551615 134630328 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/19440/statm: 18451 18141 112 34 0 18066 0
Current children cumulated CPU time (s) 1062.39
Current children cumulated vsize (KiB) 131596

[startup+1122.3 s]
/proc/loadavg: 4.02 3.98 3.62 5/204 19637
/proc/meminfo: memFree=12382524/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
[pid=19440] ppid=19422 vsize=65588 CPUtime=1119.86 cores=5,7
/proc/19440/stat : 19440 (lingeling) R 19422 19422 18696 0 -1 4202496 349717 0 0 0 111940 46 0 0 25 0 1 0 486737057 67162112 16087 33554432000 134512640 134650204 4294757856 18446744073709551615 134547453 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/19440/statm: 16397 16087 112 34 0 16012 0
Current children cumulated CPU time (s) 1122.39
Current children cumulated vsize (KiB) 123380

[startup+1182.3 s]
/proc/loadavg: 4.45 4.08 3.67 5/203 19640
/proc/meminfo: memFree=12123324/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
[pid=19440] ppid=19422 vsize=65588 CPUtime=1179.86 cores=5,7
/proc/19440/stat : 19440 (lingeling) R 19422 19422 18696 0 -1 4202496 359599 0 0 0 117939 47 0 0 25 0 1 0 486737057 67162112 16087 33554432000 134512640 134650204 4294757856 18446744073709551615 134630139 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/19440/statm: 16397 16087 112 34 0 16012 0
Current children cumulated CPU time (s) 1182.39
Current children cumulated vsize (KiB) 123380

[startup+1242.31 s]
/proc/loadavg: 4.58 4.21 3.74 5/203 19643
/proc/meminfo: memFree=12014288/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
[pid=19440] ppid=19422 vsize=66852 CPUtime=1239.86 cores=5,7
/proc/19440/stat : 19440 (lingeling) R 19422 19422 18696 0 -1 4202496 366086 0 0 0 123938 48 0 0 25 0 1 0 486737057 68456448 16403 33554432000 134512640 134650204 4294757856 18446744073709551615 134631206 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/19440/statm: 16713 16403 112 34 0 16328 0
Current children cumulated CPU time (s) 1242.39
Current children cumulated vsize (KiB) 124644



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 4.26 4.19 3.76 5/203 19645
/proc/meminfo: memFree=12075512/32951124 swapFree=67111528/67111528
[pid=19422] ppid=19420 vsize=34448 CPUtime=0.02 cores=5,7
/proc/19422/stat : 19422 (python) S 19420 19422 18696 0 -1 4202496 2082 0 0 0 2 0 0 0 18 0 1 0 486736810 35274752 1421 33554432000 4194304 5745468 140736862150912 18446744073709551615 244446945024 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/19422/statm: 8612 1421 486 379 0 932 0
[pid=19424] ppid=19422 vsize=23344 CPUtime=2.51 cores=5,7
/proc/19424/stat : 19424 (features) S 19422 19422 18696 0 -1 4202496 5128 0 0 0 245 6 0 0 25 0 1 0 486736813 23904256 3106 33554432000 134512640 135907751 4288880624 18446744073709551615 135396702 0 0 16781312 0 18446744071562362840 0 0 17 5 0 0 0
/proc/19424/statm: 5836 3106 114 341 0 5492 0
[pid=19440] ppid=19422 vsize=68088 CPUtime=1297.57 cores=5,7
/proc/19440/stat : 19440 (lingeling) R 19422 19422 18696 0 -1 4202496 378053 0 0 0 129707 50 0 0 25 0 1 0 486737057 69722112 16712 33554432000 134512640 134650204 4294757856 18446744073709551615 134524891 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/19440/statm: 17022 16712 112 34 0 16637 0
Current children cumulated CPU time (s) 1300.1
Current children cumulated vsize (KiB) 125880

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 19422 and gives
#  childrusage.ru_utime.tv_sec=2
#  childrusage.ru_utime.tv_usec=477623
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=74988
# CPU time returned by wait4() is 2.55261
# while last known CPU time is 1300.1
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1300.01
CPU time (s): 1300.1
CPU user time (s): 1299.54
CPU system time (s): 0.56
CPU usage (%): 100.007
Max. virtual memory (cumulated for all children) (KiB): 146216

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2.47762
system time used= 0.074988
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 7562
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 87
involuntary context switches= 8

runsolver used 2.01269 second user time and 5.15522 second system time

The end

Launcher Data 

Begin job on node105 at 2011-04-25 19:09:24
IDJOB=3281488
IDBENCH=53034
IDSOLVER=1618
FILE ID=node105/3281488-1303751363
RUNJOBID= node105-1303751363-19394
PBS_JOBID= 13154057
Free space on /tmp= 71804 MiB

SOLVER NAME= Coach 2011-03-18
BENCH NAME= SAT_RACE08/Q1/SAT_dat.k85.cnf
COMMAND LINE= python portfolio_solver.py -t TIMELIMIT -s RANDOMSEED  BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3281488-1303751363/watcher-3281488-1303751363 -o /tmp/evaluation-result-3281488-1303751363/solver-3281488-1303751363 -C 2400 -W 1300 -M 7680  python portfolio_solver.py -t 2400 -s 855434744  HOME/instance-3281488-1303751363.cnf

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= dce41996d9c5153aa0c253fa128d227c
RANDOM SEED=855434744

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      16514392 kB
Buffers:       2277900 kB
Cached:       13505044 kB
SwapCached:          0 kB
Active:        8463332 kB
Inactive:      7402760 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      16514392 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:          148120 kB
Writeback:           0 kB
AnonPages:       82192 kB
Mapped:          16828 kB
Slab:           506836 kB
PageTables:       5196 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   231428 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 68708 MiB
End job on node105 at 2011-04-25 19:31:06