Trace number 3386536

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-random (fixed)? (MO) 3548.2 3534.34

General information on the benchmark

Namerandom/large/
unif-k7-r85-v400-c34000-S927774663-037-UNKNOWN.cnf
MD5SUM950830e22ef92fb7f22c5b3c987a21f5
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables400
Number of clauses34000
Sum of the clauses size238000
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 534000

Solver Data 

0.35/0.63	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.35/0.64	c INFO: solving HOME/instance-3386536-1305586343.cnf
0.35/0.64	c INFO: computing task features
0.54/0.87	c INFO: computed variable-clause graph statistics
0.54/0.87	c INFO: computed clause balance statistics
1.75/2.10	c INFO: computed variable balance statistics
2.24/2.59	c INFO: computed small-clause counts
2.24/2.59	c INFO: computed Horn-clause counts
2.34/2.63	c INFO: computed variable graph statistics
2.34/2.63	c INFO: collected features for HOME/instance-3386536-1305586343.cnf in 1.97 s
2.34/2.66	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.38)
100.17/100.19	c INFO: running gnovelty+2@0 for 97 with 4900 remaining (b = 0.13)
199.04/198.67	c INFO: running adaptg2wsat2009++@0 for 97 with 4802 remaining (b = 0.08)
300.96/300.17	c INFO: running adaptg2wsat2009++@0 for 97 with 4700 remaining (b = 0.07)
401.63/400.59	c INFO: running adaptg2wsat2009++@0 for 97 with 4600 remaining (b = 0.05)
502.53/501.03	c INFO: running gnovelty+2@0 for 97 with 4499 remaining (b = 0.04)
604.42/602.58	c INFO: running FH@0 for 97 with 4398 remaining (b = 0.03)
705.24/703.01	c INFO: running adaptg2wsat2009++@0 for 97 with 4298 remaining (b = 0.03)
802.83/800.48	c INFO: running adaptg2wsat2009++@0 for 97 with 4200 remaining (b = 0.02)
904.80/901.98	c INFO: running FH@0 for 97 with 4099 remaining (b = 0.02)
1004.67/1001.45	c INFO: running hybridGM3@0 for 194 with 3999 remaining (b = 0.04)
1201.37/1197.38	c INFO: running gnovelty+2@0 for 97 with 3803 remaining (b = 0.02)
1299.41/1294.90	c INFO: running hybridGM3@0 for 194 with 3706 remaining (b = 0.03)
1495.10/1489.89	c INFO: running adaptg2wsat2009++@0 for 97 with 3511 remaining (b = 0.01)
1593.98/1588.38	c INFO: running hybridGM3@0 for 194 with 3413 remaining (b = 0.03)
1790.69/1784.36	c INFO: running hybridGM3@0 for 194 with 3217 remaining (b = 0.02)
1987.54/1980.36	c INFO: running TNM@0 for 1552 with 3021 remaining (b = 0.18)
3548.09/3534.11	c INFO: running FH@0 for 97 with 1467 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386536-1305586343/watcher-3386536-1305586343 -o /tmp/evaluation-result-3386536-1305586343/solver-3386536-1305586343 -C 5000 -W 5100 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386536-1305586343.cnf -seed 2044358801 -budget 5000 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 5030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.96 2.97 3.08 4/519 10895
/proc/meminfo: memFree=13476580/32951124 swapFree=67111348/67111528
[pid=10895] ppid=10893 vsize=15772 CPUtime=0 cores=0,2,4,6
/proc/10895/stat : 10895 (python) R 10893 10895 10801 0 -1 4202496 857 0 0 0 0 0 0 0 18 0 1 0 670232036 16150528 740 33554432000 4194304 5683644 140733737446368 18446744073709551615 212687675024 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/10895/statm: 3943 740 335 364 0 430 0

[startup+0.0711881 s]
/proc/loadavg: 3.96 2.97 3.08 4/519 10895
/proc/meminfo: memFree=13476580/32951124 swapFree=67111348/67111528
[pid=10895] ppid=10893 vsize=38904 CPUtime=0.03 cores=0,2,4,6
/proc/10895/stat : 10895 (python) S 10893 10895 10801 0 -1 4202496 1628 0 0 0 3 0 0 0 17 0 1 0 670232036 39837696 1458 33554432000 4194304 5683644 140733737446368 18446744073709551615 212687673344 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/10895/statm: 9726 1458 455 364 0 1026 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 38904

[startup+0.101272 s]
/proc/loadavg: 3.96 2.97 3.08 4/519 10895
/proc/meminfo: memFree=13476580/32951124 swapFree=67111348/67111528
[pid=10895] ppid=10893 vsize=93960 CPUtime=0.06 cores=0,2,4,6
/proc/10895/stat : 10895 (python) S 10893 10895 10801 0 -1 4202496 2693 0 0 0 5 1 0 0 17 0 1 0 670232036 96215040 2407 33554432000 4194304 5683644 140733737446368 18446744073709551615 212687673344 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/10895/statm: 23490 2407 764 364 0 1622 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 93960

[startup+0.300199 s]
/proc/loadavg: 3.96 2.97 3.08 4/519 10895
/proc/meminfo: memFree=13476580/32951124 swapFree=67111348/67111528
[pid=10895] ppid=10893 vsize=135748 CPUtime=0.19 cores=0,2,4,6
/proc/10895/stat : 10895 (python) S 10893 10895 10801 0 -1 4202496 5976 933 1 0 15 4 0 0 17 0 1 0 670232036 139005952 4149 33554432000 4194304 5683644 140733737446368 18446744073709551615 212687671301 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/10895/statm: 33937 4149 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135748

[startup+0.701068 s]
/proc/loadavg: 3.96 2.97 3.08 4/519 10895
/proc/meminfo: memFree=13476580/32951124 swapFree=67111348/67111528
[pid=10895] ppid=10893 vsize=281592 CPUtime=0.45 cores=0,2,4,6
/proc/10895/stat : 10895 (python) R 10893 10895 10801 0 -1 4202496 10133 933 1 0 36 9 0 0 16 0 1 0 670232036 288350208 7692 33554432000 4194304 5683644 140733737446368 18446744073709551615 4603291 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/10895/statm: 70398 7692 1761 364 0 6322 0
Current children cumulated CPU time (s) 0.45
Current children cumulated vsize (KiB) 281592

[startup+1.50098 s]
/proc/loadavg: 3.96 2.97 3.08 5/520 10900
/proc/meminfo: memFree=13442348/32951124 swapFree=67111348/67111528
[pid=10895] ppid=10893 vsize=289500 CPUtime=1.25 cores=0,2,4,6
/proc/10895/stat : 10895 (python) R 10893 10895 10801 0 -1 4202496 14007 933 1 0 115 10 0 0 17 0 1 0 670232036 296448000 9760 33554432000 4194304 5683644 140733737446368 18446744073709551615 4539738 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/10895/statm: 72375 9760 1826 364 0 8299 0
Current children cumulated CPU time (s) 1.25
Current children cumulated vsize (KiB) 289500

[startup+3.10069 s]
/proc/loadavg: 3.96 2.98 3.08 5/520 10900
/proc/meminfo: memFree=13440240/32951124 swapFree=67111348/67111528
[pid=10895] ppid=10893 vsize=291936 CPUtime=2.4 cores=0,2,4,6
/proc/10895/stat : 10895 (python) S 10893 10895 10801 0 -1 4202496 14335 933 1 0 230 10 0 0 21 0 2 0 670232036 298942464 7857 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/10895/statm: 72984 7857 1877 364 0 8902 0
Current children cumulated CPU time (s) 2.4
Current children cumulated vsize (KiB) 291936
heavy processes:

[startup+6.3011 s]
/proc/loadavg: 3.96 2.98 3.08 5/527 10907
/proc/meminfo: memFree=13430140/32951124 swapFree=67111348/67111528
[pid=10895] ppid=10893 vsize=291936 CPUtime=2.4 cores=0,2,4,6
/proc/10895/stat : 10895 (python) S 10893 10895 10801 0 -1 4202496 14335 933 1 0 230 10 0 0 21 0 2 0 670232036 298942464 7857 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/10895/statm: 72984 7857 1877 364 0 8902 0
[pid=10901] ppid=10895 vsize=282244 CPUtime=0 cores=0,2,4,6
/proc/10901/stat : 10901 (python) S 10895 10895 10801 0 -1 4202560 742 0 0 0 0 0 0 0 18 0 1 0 670232303 289017856 6332 33554432000 4194304 5683644 140733737446368 18446744073709551615 212687695375 0 0 16781312 514 3152116772442702082 0 0 17 4 0 0 0
/proc/10901/statm: 70561 6332 355 364 0 6479 0
[pid=10903] ppid=10901 vsize=1020188 CPUtime=3.59 cores=0,2,4,6
/proc/10903/stat : 10903 (gnovelty+2) R 10901 10903 10903 0 -1 4202496 1141 0 0 0 359 0 0 0 25 0 1 0 670232303 1044672512 877 33554432000 134512640 135663704 4287654176 18446744073709551615 134524945 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/10903/statm: 255047 877 113 282 0 254763 0
[pid=10895/tid=10902] ppid=10893 vsize=291936 CPUtime=0 cores=0,2,4,6
/proc/10895/task/10902/stat : 10902 (python) S 10893 10895 10801 0 -1 4202560 28 933 0 0 0 0 0 0 21 0 2 0 670232303 298942464 7857 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 5.99
Current children cumulated vsize (KiB) 1594368

[startup+12.7009 s]
/proc/loadavg: 3.97 3.02 3.09 5/527 10907
/proc/meminfo: memFree=13429472/32951124 swapFree=67111348/67111528
[pid=10895] ppid=10893 vsize=291936 CPUtime=2.4 cores=0,2,4,6
/proc/10895/stat : 10895 (python) S 10893 10895 10801 0 -1 4202496 14335 933 1 0 230 10 0 0 21 0 2 0 670232036 298942464 7857 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/10895/statm: 72984 7857 1877 364 0 8902 0
[pid=10901] ppid=10895 vsize=282244 CPUtime=0.02 cores=0,2,4,6
/proc/10901/stat : 10901 (python) S 10895 10895 10801 0 -1 4202560 889 0 0 0 1 1 0 0 18 0 1 0 670232303 289017856 6355 33554432000 4194304 5683644 140733737446368 18446744073709551615 212687695375 0 0 16781312 514 3152116772442702082 0 0 17 6 0 0 0
/proc/10901/statm: 70561 6355 378 364 0 6479 0
[pid=10903] ppid=10901 vsize=1020188 CPUtime=9.99 cores=0,2,4,6
/proc/10903/stat : 10903 (gnovelty+2) R 10901 10903 10903 0 -1 4202496 1141 0 0 0 999 0 0 0 25 0 1 0 670232303 1044672512 877 33554432000 134512640 135663704 4287654176 18446744073709551615 134525157 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/10903/statm: 255047 877 113 282 0 254763 0
[pid=10895/tid=10902] ppid=10893 vsize=291936 CPUtime=0 cores=0,2,4,6
/proc/10895/task/10902/stat : 10902 (python) S 10893 10895 10801 0 -1 4202560 28 933 0 0 0 0 0 0 21 0 2 0 670232303 298942464 7857 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 12.41
Current children cumulated vsize (KiB) 1594368

[startup+25.5006 s]
/proc/loadavg: 3.97 3.05 3.10 5/527 10907
/proc/meminfo: memFree=13430712/32951124 swapFree=67111348/67111528
[pid=10895] ppid=10893 vsize=291936 CPUtime=2.4 cores=0,2,4,6
/proc/10895/stat : 10895 (python) S 10893 10895 10801 0 -1 4202496 14335 933 1 0 230 10 0 0 21 0 2 0 670232036 298942464 7857 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/10895/statm: 72984 7857 1877 364 0 8902 0
[pid=10901] ppid=10895 vsize=282244 CPUtime=0.08 cores=0,2,4,6
/proc/10901/stat : 10901 (python) S 10895 10895 10801 0 -1 4202560 900 0 0 0 4 4 0 0 18 0 1 0 670232303 289017856 6355 33554432000 4194304 5683644 140733737446368 18446744073709551615 212687695375 0 0 16781312 514 1629599377666310402 0 0 17 6 0 0 0
/proc/10901/statm: 70561 6355 378 364 0 6479 0
[pid=10903] ppid=10901 vsize=1020188 CPUtime=22.79 cores=0,2,4,6
/proc/10903/stat : 10903 (gnovelty+2) R 10901 10903 10903 0 -1 4202496 1141 0 0 0 2279 0 0 0 25 0 1 0 670232303 1044672512 877 33554432000 134512640 135663704 4287654176 18446744073709551615 134524503 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/10903/statm: 255047 877 113 282 0 254763 0
[pid=10895/tid=10902] ppid=10893 vsize=291936 CPUtime=0 cores=0,2,4,6
/proc/10895/task/10902/stat : 10902 (python) S 10893 10895 10801 0 -1 4202560 28 933 0 0 0 0 0 0 21 0 2 0 670232303 298942464 7857 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 25.27
Current children cumulated vsize (KiB) 1594368

[startup+51.1009 s]
/proc/loadavg: 3.98 3.12 3.12 5/527 10908
/proc/meminfo: memFree=13430568/32951124 swapFree=67111348/67111528
[pid=10895] ppid=10893 vsize=291936 CPUtime=2.4 cores=0,2,4,6
/proc/10895/stat : 10895 (python) S 10893 10895 10801 0 -1 4202496 14335 933 1 0 230 10 0 0 21 0 2 0 670232036 298942464 7857 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/10895/statm: 72984 7857 1877 364 0 8902 0
[pid=10901] ppid=10895 vsize=282244 CPUtime=0.18 cores=0,2,4,6
/proc/10901/stat : 10901 (python) S 10895 10895 10801 0 -1 4202560 911 0 0 0 9 9 0 0 18 0 1 0 670232303 289017856 6355 33554432000 4194304 5683644 140733737446368 18446744073709551615 212687695375 0 0 16781312 514 1629599377666310402 0 0 17 6 0 0 0
/proc/10901/statm: 70561 6355 378 364 0 6479 0
[pid=10903] ppid=10901 vsize=1020188 CPUtime=48.39 cores=0,2,4,6
/proc/10903/stat : 10903 (gnovelty+2) R 10901 10903 10903 0 -1 4202496 1141 0 0 0 4839 0 0 0 25 0 1 0 670232303 1044672512 877 33554432000 134512640 135663704 4287654176 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/10903/statm: 255047 877 113 282 0 254763 0
[pid=10895/tid=10902] ppid=10893 vsize=291936 CPUtime=0 cores=0,2,4,6
/proc/10895/task/10902/stat : 10902 (python) S 10893 10895 10801 0 -1 4202560 28 933 0 0 0 0 0 0 21 0 2 0 670232303 298942464 7857 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 50.97
Current children cumulated vsize (KiB) 1594368

[startup+102.32 s]
/proc/loadavg: 3.99 3.27 3.17 5/531 10913
/proc/meminfo: memFree=13422832/32951124 swapFree=67111348/67111528
[pid=10895] ppid=10893 vsize=302348 CPUtime=2.44 cores=0,2,4,6
/proc/10895/stat : 10895 (python) S 10893 10895 10801 0 -1 4202496 15159 933 1 0 233 11 0 0 20 0 3 0 670232036 309604352 7903 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/10895/statm: 75587 7903 1880 364 0 11502 0
[pid=10901] ppid=10895 vsize=292488 CPUtime=0.38 cores=0,2,4,6
/proc/10901/stat : 10901 (python) S 10895 10895 10801 0 -1 4202560 1089 0 0 0 19 19 0 0 19 0 2 0 670232303 299507712 6392 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/10901/statm: 73122 6392 410 364 0 9040 0
[pid=10903] ppid=10901 vsize=1020188 CPUtime=97.46 cores=0,2,4,6
/proc/10903/stat : 10903 (gnovelty+2) T 10901 10903 10903 0 -1 4202496 1141 0 0 0 9746 0 0 0 25 0 1 0 670232303 1044672512 877 33554432000 134512640 135663704 4287654176 18446744073709551615 134525699 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/10903/statm: 255047 877 113 282 0 254763 0
[pid=10901/tid=10910] ppid=10895 vsize=292488 CPUtime=0 cores=0,2,4,6
/proc/10901/task/10910/stat : 10910 (python) S 10895 10895 10801 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 670242053 299507712 6392 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 514 18446604449351635328 0 0 -1 2 0 0 0
[pid=10911] ppid=10895 vsize=293128 CPUtime=0 cores=0,2,4,6
/proc/10911/stat : 10911 (python) S 10895 10895 10801 0 -1 4202560 746 0 0 0 0 0 0 0 18 0 1 0 670242056 300163072 6369 33554432000 4194304 5683644 140733737446368 18446744073709551615 212687695430 0 0 16781312 514 3545055015297057026 0 0 17 6 0 0 0
/proc/10911/statm: 73282 6369 349 364 0 9197 0
[pid=10913] ppid=10911 vsize=1020188 CPUtime=2.12 cores=0,2,4,6
/proc/10913/stat : 10913 (gnovelty+2) R 10911 10913 10913 0 -1 4202496 1141 0 0 0 212 0 0 0 25 0 1 0 670242056 1044672512 877 33554432000 134512640 135663704 4288505952 18446744073709551615 134544204 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/10913/statm: 255047 877 113 282 0 254763 0
[pid=10895/tid=10902] ppid=10893 vsize=302348 CPUtime=0 cores=0,2,4,6
/proc/10895/task/10902/stat : 10902 (python) S 10893 10895 10801 0 -1 4202560 28 933 0 0 0 0 0 0 21 0 3 0 670232303 309604352 7903 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 4 0 0 0
[pid=10895/tid=10912] ppid=10893 vsize=302348 CPUtime=0 cores=0,2,4,6
/proc/10895/task/10912/stat : 10912 (python) S 10893 10895 10801 0 -1 4202560 29 933 0 0 0 0 0 0 20 0 3 0 670242056 309604352 7903 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 102.4
Current children cumulated vsize (KiB) 2928340

[startup+162.301 s]
/proc/loadavg: 4.17 3.44 3.24 5/535 10919
/proc/meminfo: memFree=13406660/32951124 swapFree=67111348/67111528
[pid=10895] ppid=10893 vsize=302348 CPUtime=2.44 cores=0,2,4,6
/proc/10895/stat : 10895 (python) S 10893 10895 10801 0 -1 4202496 15159 933 1 0 233 11 0 0 20 0 3 0 670232036 309604352 7903 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/10895/statm: 75587 7903 1880 364 0 11502 0
[pid=10901] ppid=10895 vsize=292488 CPUtime=0.38 cores=0,2,4,6
/proc/10901/stat : 10901 (python) S 10895 10895 10801 0 -1 4202560 1089 0 0 0 19 19 0 0 19 0 2 0 670232303 299507712 6392 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/10901/statm: 73122 6392 410 364 0 9040 0
[pid=10903] ppid=10901 vsize=1020188 CPUtime=97.46 cores=0,2,4,6
/proc/10903/stat : 10903 (gnovelty+2) T 10901 10903 10903 0 -1 4202496 1141 0 0 0 9746 0 0 0 25 0 1 0 670232303 1044672512 877 33554432000 134512640 135663704 4287654176 18446744073709551615 134525699 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/10903/statm: 255047 877 113 282 0 254763 0
[pid=10901/tid=10910] ppid=10895 vsize=292488 CPUtime=0 cores=0,2,4,6
/proc/10901/task/10910/stat : 10910 (python) S 10895 10895 10801 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 670242053 299507712 6392 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 514 18446604449351635328 0 0 -1 2 0 0 0
[pid=10911] ppid=10895 vsize=293128 CPUtime=0.23 cores=0,2,4,6
/proc/10911/stat : 10911 (python) S 10895 10895 10801 0 -1 4202560 912 0 0 0 12 11 0 0 18 0 1 0 670242056 300163072 6392 33554432000 4194304 5683644 140733737446368 18446744073709551615 212687695430 0 0 16781312 514 1629599377666310402 0 0 17 6 0 0 0
/proc/10911/statm: 73282 6392 372 364 0 9197 0
[pid=10913] ppid=10911 vsize=1020188 CPUtime=62.1 cores=0,2,4,6
/proc/10913/stat : 10913 (gnovelty+2) R 10911 10913 10913 0 -1 4202496 1141 0 0 0 6210 0 0 0 25 0 1 0 670242056 1044672512 877 33554432000 134512640 135663704 4288505952 18446744073709551615 134524916 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/10913/statm: 255047 877 113 282 0 254763 0
[pid=10895/tid=10902] ppid=10893 vsize=302348 CPUtime=0 cores=0,2,4,6
/proc/10895/task/10902/stat : 10902 (python) S 10893 10895 10801 0 -1 4202560 28 933 0 0 0 0 0 0 21 0 3 0 670232303 309604352 7903 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 4 0 0 0

################
# More data... #
################

/proc/10935/statm: 105451 1871 59 146 0 104327 0
[pid=10933/tid=10942] ppid=10895 vsize=313408 CPUtime=0 cores=0,2,4,6
/proc/10933/task/10942/stat : 10942 (python) S 10895 10895 10801 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 670272093 320929792 6564 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 514 18446604440637462592 0 0 -1 6 0 0 0
[pid=10943] ppid=10895 vsize=323624 CPUtime=0.4 cores=0,2,4,6
/proc/10943/stat : 10943 (python) S 10895 10895 10801 0 -1 4202560 1114 0 0 0 20 20 0 0 19 0 2 0 670272096 331390976 6568 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/10943/statm: 80906 6568 400 364 0 16812 0
[pid=10945] ppid=10943 vsize=421804 CPUtime=100.38 cores=0,2,4,6
/proc/10945/stat : 10945 (adaptg2wsat2009) T 10943 10945 10945 0 -1 4202496 2140 0 0 0 10038 0 0 0 25 0 1 0 670272096 431927296 1871 33554432000 134512640 135107411 4288363952 18446744073709551615 134532347 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/10945/statm: 105451 1871 59 146 0 104327 0
[pid=10943/tid=10953] ppid=10895 vsize=323624 CPUtime=0 cores=0,2,4,6
/proc/10943/task/10953/stat : 10953 (python) S 10895 10895 10801 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 670282136 331390976 6568 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 514 18446604446017864832 0 0 -1 6 0 0 0
[pid=10954] ppid=10895 vsize=333984 CPUtime=0.42 cores=0,2,4,6
/proc/10954/stat : 10954 (python) S 10895 10895 10801 0 -1 4202560 1123 0 0 0 23 19 0 0 19 0 2 0 670282139 341999616 6572 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/10954/statm: 83496 6572 400 364 0 19399 0
[pid=10956] ppid=10954 vsize=1020188 CPUtime=101.5 cores=0,2,4,6
/proc/10956/stat : 10956 (gnovelty+2) T 10954 10956 10956 0 -1 4202496 1143 0 0 0 10150 0 0 0 25 0 1 0 670282140 1044672512 876 33554432000 134512640 135663704 4288920128 18446744073709551615 134524894 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/10956/statm: 255047 876 113 282 0 254763 0
[pid=10954/tid=10977] ppid=10895 vsize=333984 CPUtime=0 cores=0,2,4,6
/proc/10954/task/10977/stat : 10977 (python) S 10895 10895 10801 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 670292291 341999616 6572 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 514 18446604450135869376 0 0 -1 6 0 0 0
[pid=10978] ppid=10895 vsize=344248 CPUtime=0.4 cores=0,2,4,6
/proc/10978/stat : 10978 (python) S 10895 10895 10801 0 -1 4202560 1182 0 0 0 20 20 0 0 19 0 2 0 670292294 352509952 6582 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/10978/statm: 86062 6582 406 364 0 21962 0
[pid=10980] ppid=10978 vsize=251288 CPUtime=100.3 cores=0,2,4,6
/proc/10980/stat : 10980 (FH) T 10978 10980 10980 0 -1 4202496 1962 0 0 0 10030 0 0 0 25 0 1 0 670292295 257318912 1693 33554432000 134512640 134936988 4294004768 18446744073709551615 134552382 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/10980/statm: 62822 1693 41 104 0 62326 0
[pid=10978/tid=10987] ppid=10895 vsize=344248 CPUtime=0 cores=0,2,4,6
/proc/10978/task/10987/stat : 10987 (python) S 10895 10895 10801 0 -1 4202560 94 0 0 0 0 0 0 0 19 0 2 0 670302334 352509952 6582 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 514 18446604445345006592 0 0 -1 4 0 0 0
[pid=10988] ppid=10895 vsize=354384 CPUtime=0.42 cores=0,2,4,6
/proc/10988/stat : 10988 (python) S 10895 10895 10801 0 -1 4202560 1176 0 0 0 23 19 0 0 19 0 2 0 670302338 362889216 6586 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/10988/statm: 88596 6586 406 364 0 24493 0
[pid=10990] ppid=10988 vsize=421804 CPUtime=97.24 cores=0,2,4,6
/proc/10990/stat : 10990 (adaptg2wsat2009) T 10988 10990 10990 0 -1 4202496 2146 0 0 0 9724 0 0 0 25 0 1 0 670302338 431927296 1873 33554432000 134512640 135107411 4293903264 18446744073709551615 134531707 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/10990/statm: 105451 1873 59 146 0 104327 0
[pid=10988/tid=11001] ppid=10895 vsize=354384 CPUtime=0 cores=0,2,4,6
/proc/10988/task/11001/stat : 11001 (python) S 10895 10895 10801 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 670312081 362889216 6586 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 514 18446604446430687424 0 0 -1 2 0 0 0
[pid=11002] ppid=10895 vsize=365248 CPUtime=0.44 cores=0,2,4,6
/proc/11002/stat : 11002 (python) S 10895 10895 10801 0 -1 4202560 1134 0 0 0 25 19 0 0 19 0 2 0 670312085 374013952 6584 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11002/statm: 91312 6584 400 364 0 27206 0
[pid=11004] ppid=11002 vsize=421804 CPUtime=101.45 cores=0,2,4,6
/proc/11004/stat : 11004 (adaptg2wsat2009) T 11002 11004 11004 0 -1 4202496 2150 0 0 0 10145 0 0 0 25 0 1 0 670312085 431927296 1871 33554432000 134512640 135107411 4287782592 18446744073709551615 134531935 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/11004/statm: 105451 1871 59 146 0 104327 0
[pid=11002/tid=11008] ppid=10895 vsize=365248 CPUtime=0 cores=0,2,4,6
/proc/11002/task/11008/stat : 11008 (python) S 10895 10895 10801 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 670322231 374013952 6584 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 514 18446604450577278144 0 0 -1 2 0 0 0
[pid=11009] ppid=10895 vsize=375296 CPUtime=0.42 cores=0,2,4,6
/proc/11009/stat : 11009 (python) S 10895 10895 10801 0 -1 4202560 1138 0 0 0 25 17 0 0 19 0 2 0 670322235 384303104 6670 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11009/statm: 93824 6670 400 364 0 29715 0
[pid=11011] ppid=11009 vsize=251292 CPUtime=99.41 cores=0,2,4,6
/proc/11011/stat : 11011 (FH) T 11009 11011 11011 0 -1 4202496 1962 0 0 0 9941 0 0 0 25 0 1 0 670322235 257323008 1693 33554432000 134512640 134936988 4289182352 18446744073709551615 134554252 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/11011/statm: 62823 1693 41 104 0 62327 0
[pid=11009/tid=11018] ppid=10895 vsize=375296 CPUtime=0 cores=0,2,4,6
/proc/11009/task/11018/stat : 11018 (python) S 10895 10895 10801 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 670332178 384303104 6670 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 514 18446604448629426176 0 0 -1 6 0 0 0
[pid=11019] ppid=10895 vsize=385276 CPUtime=0.81 cores=0,2,4,6
/proc/11019/stat : 11019 (python) S 10895 10895 10801 0 -1 4202560 1161 0 0 0 42 39 0 0 19 0 2 0 670332182 394522624 6673 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11019/statm: 96319 6673 399 364 0 32207 0
[pid=11021] ppid=11019 vsize=268424 CPUtime=195.88 cores=0,2,4,6
/proc/11021/stat : 11021 (hybridGM3) T 11019 11021 11021 0 -1 4202496 5651 0 0 0 19588 0 0 0 25 0 1 0 670332182 274866176 5377 33554432000 134512640 135204385 4290379520 18446744073709551615 134537704 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/11021/statm: 67106 5377 63 169 0 66936 0
[pid=11019/tid=11031] ppid=10895 vsize=385276 CPUtime=0 cores=0,2,4,6
/proc/11019/task/11031/stat : 11031 (python) S 10895 10895 10801 0 -1 4202560 51 0 0 0 0 0 0 0 19 0 2 0 670351771 394522624 6673 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 514 18446604443727045824 0 0 -1 6 0 0 0
[pid=11032] ppid=10895 vsize=396160 CPUtime=0.42 cores=0,2,4,6
/proc/11032/stat : 11032 (python) S 10895 10895 10801 0 -1 4202560 1149 0 0 0 23 19 0 0 19 0 2 0 670351775 405667840 6678 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11032/statm: 99040 6678 400 364 0 34925 0
[pid=11034] ppid=11032 vsize=1020188 CPUtime=97.47 cores=0,2,4,6
/proc/11034/stat : 11034 (gnovelty+2) T 11032 11034 11034 0 -1 4202496 1148 0 0 0 9747 0 0 0 25 0 1 0 670351775 1044672512 876 33554432000 134512640 135663704 4291533552 18446744073709551615 134524918 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/11034/statm: 255047 876 113 282 0 254763 0
[pid=11032/tid=11042] ppid=10895 vsize=396160 CPUtime=0 cores=0,2,4,6
/proc/11032/task/11042/stat : 11042 (python) S 10895 10895 10801 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 670361523 405667840 6678 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 514 18446604445173453824 0 0 -1 6 0 0 0
[pid=11043] ppid=10895 vsize=406116 CPUtime=0.83 cores=0,2,4,6
/proc/11043/stat : 11043 (python) S 10895 10895 10801 0 -1 4202560 1172 0 0 0 44 39 0 0 19 0 2 0 670361527 415862784 6718 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11043/statm: 101529 6718 400 364 0 37411 0
[pid=11045] ppid=11043 vsize=268424 CPUtime=194.93 cores=0,2,4,6
/proc/11045/stat : 11045 (hybridGM3) T 11043 11045 11045 0 -1 4202496 5650 0 0 0 19493 0 0 0 25 0 1 0 670361527 274866176 5378 33554432000 134512640 135204385 4290411216 18446744073709551615 134538284 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/11045/statm: 67106 5378 63 169 0 66936 0
[pid=11043/tid=11055] ppid=10895 vsize=406116 CPUtime=0 cores=0,2,4,6
/proc/11043/task/11055/stat : 11055 (python) S 10895 10895 10801 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 670381022 415862784 6718 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 514 18446604447368893696 0 0 -1 6 0 0 0
[pid=11056] ppid=10895 vsize=416536 CPUtime=0.42 cores=0,2,4,6
/proc/11056/stat : 11056 (python) S 10895 10895 10801 0 -1 4202560 1152 0 0 0 23 19 0 0 19 0 2 0 670381026 426532864 6722 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11056/statm: 104134 6722 400 364 0 40013 0
[pid=11058] ppid=11056 vsize=421804 CPUtime=98.42 cores=0,2,4,6
/proc/11058/stat : 11058 (adaptg2wsat2009) T 11056 11058 11058 0 -1 4202496 2147 0 0 0 9842 0 0 0 25 0 1 0 670381026 431927296 1873 33554432000 134512640 135107411 4293292272 18446744073709551615 134531768 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/11058/statm: 105451 1873 59 146 0 104327 0
[pid=11056/tid=11066] ppid=10895 vsize=416536 CPUtime=0 cores=0,2,4,6
/proc/11056/task/11066/stat : 11066 (python) S 10895 10895 10801 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 670390871 426532864 6722 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 514 18446604448717141120 0 0 -1 6 0 0 0
[pid=11067] ppid=10895 vsize=426784 CPUtime=0.85 cores=0,2,4,6
/proc/11067/stat : 11067 (python) S 10895 10895 10801 0 -1 4202560 1208 0 0 0 49 36 0 0 19 0 2 0 670390875 437026816 6733 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11067/statm: 106696 6733 407 364 0 42572 0
[pid=11069] ppid=11067 vsize=268424 CPUtime=195.81 cores=0,2,4,6
/proc/11069/stat : 11069 (hybridGM3) T 11067 11069 11069 0 -1 4202496 5652 0 0 0 19581 0 0 0 25 0 1 0 670390875 274866176 5378 33554432000 134512640 135204385 4287056304 18446744073709551615 134538000 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/11069/statm: 67106 5378 63 169 0 66936 0
[pid=11067/tid=11079] ppid=10895 vsize=426784 CPUtime=0 cores=0,2,4,6
/proc/11067/task/11079/stat : 11079 (python) S 10895 10895 10801 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 670410469 437026816 6733 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 514 18446604441372449792 0 0 -1 2 0 0 0
[pid=11080] ppid=10895 vsize=437220 CPUtime=0.85 cores=0,2,4,6
/proc/11080/stat : 11080 (python) S 10895 10895 10801 0 -1 4202560 1226 0 0 0 46 39 0 0 19 0 2 0 670410473 447713280 6737 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11080/statm: 109305 6737 407 364 0 45178 0
[pid=11082] ppid=11080 vsize=268424 CPUtime=195.96 cores=0,2,4,6
/proc/11082/stat : 11082 (hybridGM3) T 11080 11082 11082 0 -1 4202496 5649 0 0 0 19595 1 0 0 25 0 1 0 670410473 274866176 5377 33554432000 134512640 135204385 4289224416 18446744073709551615 134538041 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/11082/statm: 67106 5377 63 169 0 66936 0
[pid=11080/tid=11088] ppid=10895 vsize=437220 CPUtime=0 cores=0,2,4,6
/proc/11080/task/11088/stat : 11088 (python) S 10895 10895 10801 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 670430069 447713280 6737 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 514 18446604443727044928 0 0 -1 4 0 0 0
[pid=11089] ppid=10895 vsize=447484 CPUtime=6.71 cores=0,2,4,6
/proc/11089/stat : 11089 (python) S 10895 10895 10801 0 -1 4202560 1210 0 0 0 357 314 0 0 19 0 2 0 670430073 458223616 6744 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11089/statm: 111871 6744 401 364 0 47741 0
[pid=11091] ppid=11089 vsize=459768 CPUtime=1553.74 cores=0,2,4,6
/proc/11091/stat : 11091 (TNM) T 11089 11091 11091 0 -1 4202496 1934 0 0 0 155373 1 0 0 25 0 1 0 670430073 470802432 1658 33554432000 134512640 135008748 4289130144 18446744073709551615 134527339 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/11091/statm: 114942 1658 46 122 0 113842 0
[pid=11089/tid=11213] ppid=10895 vsize=447484 CPUtime=0 cores=0,2,4,6
/proc/11089/task/11213/stat : 11213 (python) S 10895 10895 10801 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 670585446 458223616 6744 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 514 18446604443727046720 0 0 -1 6 0 0 0
[pid=11214] ppid=10895 vsize=457936 CPUtime=0 cores=0,2,4,6
/proc/11214/stat : 11214 (python) S 10895 10895 10801 0 -1 4202560 824 0 0 0 0 0 0 0 19 0 1 0 670585448 468926464 6692 33554432000 4194304 5683644 140733737446368 18446744073709551615 212687695430 0 0 16781312 514 3602951724202983423 0 0 17 4 0 0 0
/proc/11214/statm: 114484 6692 346 364 0 50351 0
[pid=11216] ppid=11214 vsize=249144 CPUtime=0.1 cores=0,2,4,6
/proc/11216/stat : 11216 (FH) R 11214 11216 11216 0 -1 4202496 1569 0 0 0 10 0 0 0 20 0 1 0 670585449 255123456 1295 33554432000 134512640 134936988 4287777328 18446744073709551615 134523683 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/11216/statm: 62286 1295 37 104 0 61790 0
[pid=10895/tid=10902] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/10902/stat : 10902 (python) S 10893 10895 10801 0 -1 4202560 28 933 0 0 0 0 0 0 21 0 19 0 670232303 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 4 0 0 0
[pid=10895/tid=10912] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/10912/stat : 10912 (python) S 10893 10895 10801 0 -1 4202560 29 933 0 0 0 0 0 0 20 0 19 0 670242056 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 2 0 0 0
[pid=10895/tid=10923] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/10923/stat : 10923 (python) S 10893 10895 10801 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 19 0 670251904 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 2 0 0 0
[pid=10895/tid=10934] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/10934/stat : 10934 (python) S 10893 10895 10801 0 -1 4202560 26 933 0 0 0 0 0 0 20 0 19 0 670262054 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 2 0 0 0
[pid=10895/tid=10944] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/10944/stat : 10944 (python) S 10893 10895 10801 0 -1 4202560 26 933 0 0 0 0 0 0 20 0 19 0 670272096 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 2 0 0 0
[pid=10895/tid=10955] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/10955/stat : 10955 (python) S 10893 10895 10801 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 19 0 670282139 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 2 0 0 0
[pid=10895/tid=10979] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/10979/stat : 10979 (python) S 10893 10895 10801 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 19 0 670292295 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 6 0 0 0
[pid=10895/tid=10989] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/10989/stat : 10989 (python) S 10893 10895 10801 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 19 0 670302338 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 4 0 0 0
[pid=10895/tid=11003] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/11003/stat : 11003 (python) S 10893 10895 10801 0 -1 4202560 29 933 0 0 0 0 0 0 20 0 19 0 670312085 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 6 0 0 0
[pid=10895/tid=11010] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/11010/stat : 11010 (python) S 10893 10895 10801 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 19 0 670322235 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 2 0 0 0
[pid=10895/tid=11020] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/11020/stat : 11020 (python) S 10893 10895 10801 0 -1 4202560 29 933 0 0 0 0 0 0 20 0 19 0 670332182 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 6 0 0 0
[pid=10895/tid=11033] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/11033/stat : 11033 (python) S 10893 10895 10801 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 19 0 670351775 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 2 0 0 0
[pid=10895/tid=11044] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/11044/stat : 11044 (python) S 10893 10895 10801 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 19 0 670361527 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 4 0 0 0
[pid=10895/tid=11057] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/11057/stat : 11057 (python) S 10893 10895 10801 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 19 0 670381026 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 4 0 0 0
[pid=10895/tid=11068] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/11068/stat : 11068 (python) S 10893 10895 10801 0 -1 4202560 26 933 0 0 0 0 0 0 20 0 19 0 670390875 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 6 0 0 0
[pid=10895/tid=11081] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/11081/stat : 11081 (python) S 10893 10895 10801 0 -1 4202560 26 933 0 0 0 0 0 0 20 0 19 0 670410473 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 6 0 0 0
[pid=10895/tid=11090] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/11090/stat : 11090 (python) S 10893 10895 10801 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 19 0 670430073 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 4 0 0 0
[pid=10895/tid=11215] ppid=10893 vsize=467456 CPUtime=0 cores=0,2,4,6
/proc/10895/task/11215/stat : 11215 (python) S 10893 10895 10801 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 19 0 670585448 478674944 8261 33554432000 4194304 5683644 140733737446368 18446744073709551615 212699498929 0 0 16781312 2 18446604448717140224 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3548.2
Current children cumulated vsize (KiB) 16036604

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 10895 and gives
#  childrusage.ru_utime.tv_sec=2
#  childrusage.ru_utime.tv_usec=876562
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=161975
# CPU time returned by wait4() is 3.03854
# while last known CPU time is 3548.2
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 3534.34
CPU time (s): 3548.2
CPU user time (s): 3541.05
CPU system time (s): 7.15
CPU usage (%): 100.392
Max. virtual memory (cumulated for all children) (KiB): 16036604

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2.87656
system time used= 0.161975
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 31749
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2924
involuntary context switches= 65

runsolver used 36.7324 second user time and 75.3625 second system time

The end

Launcher Data 

Begin job on node139 at 2011-05-17 00:52:23
IDJOB=3386536
IDBENCH=82601
IDSOLVER=1872
FILE ID=node139/3386536-1305586343
RUNJOBID= node139-1305586143-10819
PBS_JOBID= 13350339
Free space on /tmp= 73244 MiB

SOLVER NAME= borg-sat 11.03.02-seq-random (fixed)
BENCH NAME= SAT11/random/large/unif-k7-r85-v400-c34000-S927774663-037-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386536-1305586343/watcher-3386536-1305586343 -o /tmp/evaluation-result-3386536-1305586343/solver-3386536-1305586343 -C 5000 -W 5100 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386536-1305586343.cnf -seed 2044358801 -budget 5000

TIME LIMIT= 5000 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 950830e22ef92fb7f22c5b3c987a21f5
RANDOM SEED=2044358801

node139.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5321.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      13476612 kB
Buffers:        343356 kB
Cached:        3405044 kB
SwapCached:          0 kB
Active:       16929100 kB
Inactive:      2194244 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      13476612 kB
SwapTotal:    67111528 kB
SwapFree:     67111348 kB
Dirty:           32640 kB
Writeback:           0 kB
AnonPages:    15375084 kB
Mapped:          45244 kB
Slab:           209956 kB
PageTables:      71468 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 142900368 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73196 MiB
End job on node139 at 2011-05-17 01:51:19