Trace number 3386904

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-random (fixed)UNSAT 3542.45 3559.29

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v500-c2130-S1205196501-095.UNKNOWN.cnf
MD5SUMb1bf2be38e1cea16efc5e7effa1a10c7
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark757.265
Satisfiable
(Un)Satisfiability was proved
Number of variables500
Number of clauses2130
Sum of the clauses size6390
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 32130
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.30/0.49	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.30/0.49	c INFO: solving HOME/instance-3386904-1305594764.cnf
0.30/0.49	c INFO: computing task features
0.30/0.50	c INFO: computed variable-clause graph statistics
0.30/0.50	c INFO: computed clause balance statistics
0.30/0.54	c INFO: computed variable balance statistics
0.30/0.57	c INFO: computed small-clause counts
0.30/0.57	c INFO: computed Horn-clause counts
0.30/0.57	c INFO: computed variable graph statistics
0.30/0.57	c INFO: collected features for HOME/instance-3386904-1305594764.cnf in 0.08 s
0.46/0.60	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.57)
100.08/99.97	c INFO: running march_hi@0 for 388 with 4900 remaining (b = 0.35)
491.20/491.35	c INFO: running march_hi@0 for 97 with 4510 remaining (b = 0.12)
593.14/593.59	c INFO: running march_hi@389 for 97 with 4409 remaining (b = 0.10)
692.04/693.56	c INFO: running march_hi@488 for 194 with 4310 remaining (b = 0.18)
887.16/889.15	c INFO: running march_hi@101 for 97 with 4116 remaining (b = 0.08)
981.41/983.38	c INFO: running march_hi@195 for 194 with 4022 remaining (b = 0.16)
1178.27/1182.44	c INFO: running march_hi@0 for 97 with 3826 remaining (b = 0.11)
1278.98/1284.76	c INFO: running march_hi@391 for 97 with 3726 remaining (b = 0.09)
1378.14/1384.50	c INFO: running march_hi@490 for 194 with 3627 remaining (b = 0.15)
1568.02/1574.82	c INFO: running gnovelty+2@0 for 97 with 3438 remaining (b = 0.07)
1665.87/1674.22	c INFO: running march_hi@100 for 291 with 3340 remaining (b = 0.21)
1960.05/1973.09	c INFO: running march_hi@0 for 97 with 3047 remaining (b = 0.10)
2061.06/2075.80	c INFO: running march_hi@393 for 97 with 2947 remaining (b = 0.08)
2154.61/2170.83	c INFO: running march_hi@486 for 97 with 2854 remaining (b = 0.07)
2251.41/2269.21	c INFO: running march_hi@582 for 97 with 2757 remaining (b = 0.08)
2353.51/2373.01	c INFO: running march_hi@100 for 291 with 2656 remaining (b = 0.19)
2644.41/2662.82	c INFO: running march_hi@0 for 388 with 2366 remaining (b = 0.25)
3038.68/3057.58	c INFO: running march_hi@0 for 97 with 1973 remaining (b = 0.07)
3139.87/3158.38	c INFO: running march_hi@390 for 97 with 1872 remaining (b = 0.06)
3237.94/3256.18	c INFO: running march_hi@100 for 291 with 1775 remaining (b = 0.15)
3530.82/3547.97	c INFO: running march_hi@682 for 1483 with 1483 remaining (b = 0.47)
3542.19/3559.27	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386904-1305594764/watcher-3386904-1305594764 -o /tmp/evaluation-result-3386904-1305594764/solver-3386904-1305594764 -C 5000 -W 5100 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386904-1305594764.cnf -seed 455774198 -budget 5000 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 5030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.08 7.11 7.09 7/454 13222
/proc/meminfo: memFree=15778424/32951124 swapFree=57171160/67111528
[pid=13222] ppid=13220 vsize=84528 CPUtime=0.05 cores=1,3,5,7
/proc/13222/stat : 13222 (python) R 13220 13222 12094 0 -1 4202496 2272 0 0 0 4 1 0 0 25 0 1 0 651892040 86556672 2017 33554432000 4194304 5683644 140734586457184 18446744073709551615 270338410268 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/13222/statm: 21132 2017 633 364 0 1336 0

[startup+0.155813 s]
/proc/loadavg: 7.08 7.11 7.09 7/454 13222
/proc/meminfo: memFree=15778424/32951124 swapFree=57171160/67111528
[pid=13222] ppid=13220 vsize=116608 CPUtime=0.1 cores=1,3,5,7
/proc/13222/stat : 13222 (python) S 13220 13222 12094 0 -1 4202496 3855 0 0 0 8 2 0 0 18 0 1 0 651892040 119406592 3492 33554432000 4194304 5683644 140734586457184 18446744073709551615 270338381824 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/13222/statm: 29152 3492 895 364 0 2653 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 116608

[startup+0.209225 s]
/proc/loadavg: 7.08 7.11 7.09 7/454 13222
/proc/meminfo: memFree=15778424/32951124 swapFree=57171160/67111528
[pid=13222] ppid=13220 vsize=126664 CPUtime=0.15 cores=1,3,5,7
/proc/13222/stat : 13222 (python) S 13220 13222 12094 0 -1 4202496 5098 933 1 0 12 3 0 0 18 0 1 0 651892040 129703936 3923 33554432000 4194304 5683644 140734586457184 18446744073709551615 270338381824 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/13222/statm: 31666 3923 1003 364 0 2961 0
Current children cumulated CPU time (s) 0.15
Current children cumulated vsize (KiB) 126664

[startup+0.301234 s]
/proc/loadavg: 7.08 7.11 7.09 7/454 13222
/proc/meminfo: memFree=15778424/32951124 swapFree=57171160/67111528
[pid=13222] ppid=13220 vsize=144104 CPUtime=0.22 cores=1,3,5,7
/proc/13222/stat : 13222 (python) S 13220 13222 12094 0 -1 4202496 6253 933 1 0 16 6 0 0 18 0 1 0 651892040 147562496 4373 33554432000 4194304 5683644 140734586457184 18446744073709551615 270338381824 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/13222/statm: 36026 4373 1185 364 0 3182 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 144104

[startup+0.70043 s]
/proc/loadavg: 7.08 7.11 7.09 7/454 13222
/proc/meminfo: memFree=15778424/32951124 swapFree=57171160/67111528
[pid=13222] ppid=13220 vsize=288824 CPUtime=0.49 cores=1,3,5,7
/proc/13222/stat : 13222 (python) S 13220 13222 12094 0 -1 4202496 9832 933 1 0 40 9 0 0 19 0 2 0 651892040 295755776 7103 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/13222/statm: 72206 7103 1875 364 0 8124 0
Current children cumulated CPU time (s) 0.49
Current children cumulated vsize (KiB) 288824

[startup+1.50027 s]
/proc/loadavg: 7.15 7.12 7.09 8/458 13230
/proc/meminfo: memFree=15753840/32951124 swapFree=57171160/67111528
[pid=13222] ppid=13220 vsize=288824 CPUtime=0.49 cores=1,3,5,7
/proc/13222/stat : 13222 (python) S 13220 13222 12094 0 -1 4202496 9832 933 1 0 40 9 0 0 19 0 2 0 651892040 295755776 7103 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/13222/statm: 72206 7103 1875 364 0 8124 0
[pid=13228] ppid=13222 vsize=278580 CPUtime=0 cores=1,3,5,7
/proc/13228/stat : 13228 (python) S 13222 13222 12094 0 -1 4202560 736 0 0 0 0 0 0 0 19 0 1 0 651892100 285265920 5578 33554432000 4194304 5683644 140734586457184 18446744073709551615 270338403855 0 0 16781312 514 10229752560151953407 0 0 17 7 0 0 0
/proc/13228/statm: 69645 5578 354 364 0 5563 0
[pid=13230] ppid=13228 vsize=1017800 CPUtime=0.88 cores=1,3,5,7
/proc/13230/stat : 13230 (gnovelty+2) R 13228 13230 13230 0 -1 4202496 438 0 0 0 88 0 0 0 23 0 1 0 651892101 1042227200 177 33554432000 134512640 135663704 4288266848 18446744073709551615 134545598 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/13230/statm: 254450 177 113 282 0 254166 0
[pid=13222/tid=13229] ppid=13220 vsize=288824 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13229/stat : 13229 (python) S 13220 13222 12094 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 2 0 651892101 295755776 7103 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1.37
Current children cumulated vsize (KiB) 1585204

[startup+3.10095 s]
/proc/loadavg: 7.15 7.12 7.09 8/458 13230
/proc/meminfo: memFree=15753096/32951124 swapFree=57171160/67111528
[pid=13222] ppid=13220 vsize=288824 CPUtime=0.49 cores=1,3,5,7
/proc/13222/stat : 13222 (python) S 13220 13222 12094 0 -1 4202496 9832 933 1 0 40 9 0 0 19 0 2 0 651892040 295755776 7103 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/13222/statm: 72206 7103 1875 364 0 8124 0
[pid=13228] ppid=13222 vsize=278580 CPUtime=0 cores=1,3,5,7
/proc/13228/stat : 13228 (python) S 13222 13222 12094 0 -1 4202560 736 0 0 0 0 0 0 0 18 0 1 0 651892100 285265920 5578 33554432000 4194304 5683644 140734586457184 18446744073709551615 270338403855 0 0 16781312 514 10229752560151920901 0 0 17 7 0 0 0
/proc/13228/statm: 69645 5578 354 364 0 5563 0
[pid=13230] ppid=13228 vsize=1017800 CPUtime=2.49 cores=1,3,5,7
/proc/13230/stat : 13230 (gnovelty+2) R 13228 13230 13230 0 -1 4202496 438 0 0 0 249 0 0 0 25 0 1 0 651892101 1042227200 177 33554432000 134512640 135663704 4288266848 18446744073709551615 134525353 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/13230/statm: 254450 177 113 282 0 254166 0
[pid=13222/tid=13229] ppid=13220 vsize=288824 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13229/stat : 13229 (python) S 13220 13222 12094 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 2 0 651892101 295755776 7103 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2.98
Current children cumulated vsize (KiB) 1585204

[startup+6.30032 s]
/proc/loadavg: 7.14 7.12 7.09 8/458 13230
/proc/meminfo: memFree=15751732/32951124 swapFree=57171160/67111528
[pid=13222] ppid=13220 vsize=288824 CPUtime=0.49 cores=1,3,5,7
/proc/13222/stat : 13222 (python) S 13220 13222 12094 0 -1 4202496 9832 933 1 0 40 9 0 0 19 0 2 0 651892040 295755776 7103 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/13222/statm: 72206 7103 1875 364 0 8124 0
[pid=13228] ppid=13222 vsize=278580 CPUtime=0.01 cores=1,3,5,7
/proc/13228/stat : 13228 (python) S 13222 13222 12094 0 -1 4202560 892 0 0 0 1 0 0 0 18 0 1 0 651892100 285265920 5601 33554432000 4194304 5683644 140734586457184 18446744073709551615 270338403855 0 0 16781312 514 10229752560151920901 0 0 17 7 0 0 0
/proc/13228/statm: 69645 5601 377 364 0 5563 0
[pid=13230] ppid=13228 vsize=1017800 CPUtime=5.69 cores=1,3,5,7
/proc/13230/stat : 13230 (gnovelty+2) R 13228 13230 13230 0 -1 4202496 438 0 0 0 569 0 0 0 25 0 1 0 651892101 1042227200 177 33554432000 134512640 135663704 4288266848 18446744073709551615 134524909 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/13230/statm: 254450 177 113 282 0 254166 0
[pid=13222/tid=13229] ppid=13220 vsize=288824 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13229/stat : 13229 (python) S 13220 13222 12094 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 2 0 651892101 295755776 7103 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 6.19
Current children cumulated vsize (KiB) 1585204

[startup+12.701 s]
/proc/loadavg: 7.13 7.12 7.09 8/458 13231
/proc/meminfo: memFree=15750680/32951124 swapFree=57171160/67111528
[pid=13222] ppid=13220 vsize=288824 CPUtime=0.49 cores=1,3,5,7
/proc/13222/stat : 13222 (python) S 13220 13222 12094 0 -1 4202496 9832 933 1 0 40 9 0 0 19 0 2 0 651892040 295755776 7103 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/13222/statm: 72206 7103 1875 364 0 8124 0
[pid=13228] ppid=13222 vsize=278580 CPUtime=0.04 cores=1,3,5,7
/proc/13228/stat : 13228 (python) S 13222 13222 12094 0 -1 4202560 894 0 0 0 2 2 0 0 18 0 1 0 651892100 285265920 5601 33554432000 4194304 5683644 140734586457184 18446744073709551615 270338403855 0 0 16781312 514 17508500015402680581 0 0 17 7 0 0 0
/proc/13228/statm: 69645 5601 377 364 0 5563 0
[pid=13230] ppid=13228 vsize=1017800 CPUtime=12.09 cores=1,3,5,7
/proc/13230/stat : 13230 (gnovelty+2) R 13228 13230 13230 0 -1 4202496 438 0 0 0 1209 0 0 0 25 0 1 0 651892101 1042227200 177 33554432000 134512640 135663704 4288266848 18446744073709551615 134545631 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/13230/statm: 254450 177 113 282 0 254166 0
[pid=13222/tid=13229] ppid=13220 vsize=288824 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13229/stat : 13229 (python) S 13220 13222 12094 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 2 0 651892101 295755776 7103 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.62
Current children cumulated vsize (KiB) 1585204

[startup+25.5005 s]
/proc/loadavg: 7.10 7.11 7.09 8/458 13231
/proc/meminfo: memFree=15748616/32951124 swapFree=57171160/67111528
[pid=13222] ppid=13220 vsize=288824 CPUtime=0.49 cores=1,3,5,7
/proc/13222/stat : 13222 (python) S 13220 13222 12094 0 -1 4202496 9832 933 1 0 40 9 0 0 19 0 2 0 651892040 295755776 7103 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/13222/statm: 72206 7103 1875 364 0 8124 0
[pid=13228] ppid=13222 vsize=278580 CPUtime=0.07 cores=1,3,5,7
/proc/13228/stat : 13228 (python) S 13222 13222 12094 0 -1 4202560 897 0 0 0 3 4 0 0 18 0 1 0 651892100 285265920 5601 33554432000 4194304 5683644 140734586457184 18446744073709551615 270338403855 0 0 16781312 514 17508500015402680581 0 0 17 7 0 0 0
/proc/13228/statm: 69645 5601 377 364 0 5563 0
[pid=13230] ppid=13228 vsize=1017800 CPUtime=24.89 cores=1,3,5,7
/proc/13230/stat : 13230 (gnovelty+2) R 13228 13230 13230 0 -1 4202496 438 0 0 0 2489 0 0 0 25 0 1 0 651892101 1042227200 177 33554432000 134512640 135663704 4288266848 18446744073709551615 134524551 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/13230/statm: 254450 177 113 282 0 254166 0
[pid=13222/tid=13229] ppid=13220 vsize=288824 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13229/stat : 13229 (python) S 13220 13222 12094 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 2 0 651892101 295755776 7103 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 25.45
Current children cumulated vsize (KiB) 1585204

[startup+51.1004 s]
/proc/loadavg: 7.07 7.10 7.09 8/458 13232
/proc/meminfo: memFree=15742172/32951124 swapFree=57171160/67111528
[pid=13222] ppid=13220 vsize=288824 CPUtime=0.49 cores=1,3,5,7
/proc/13222/stat : 13222 (python) S 13220 13222 12094 0 -1 4202496 9832 933 1 0 40 9 0 0 19 0 2 0 651892040 295755776 7103 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/13222/statm: 72206 7103 1875 364 0 8124 0
[pid=13228] ppid=13222 vsize=278580 CPUtime=0.15 cores=1,3,5,7
/proc/13228/stat : 13228 (python) S 13222 13222 12094 0 -1 4202560 906 0 0 0 7 8 0 0 18 0 1 0 651892100 285265920 5601 33554432000 4194304 5683644 140734586457184 18446744073709551615 270338403855 0 0 16781312 514 17508500015402680581 0 0 17 7 0 0 0
/proc/13228/statm: 69645 5601 377 364 0 5563 0
[pid=13230] ppid=13228 vsize=1017800 CPUtime=50.49 cores=1,3,5,7
/proc/13230/stat : 13230 (gnovelty+2) R 13228 13230 13230 0 -1 4202496 438 0 0 0 5049 0 0 0 25 0 1 0 651892101 1042227200 177 33554432000 134512640 135663704 4288266848 18446744073709551615 134524498 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/13230/statm: 254450 177 113 282 0 254166 0
[pid=13222/tid=13229] ppid=13220 vsize=288824 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13229/stat : 13229 (python) S 13220 13222 12094 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 2 0 651892101 295755776 7103 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 51.13
Current children cumulated vsize (KiB) 1585204

[startup+102.301 s]
/proc/loadavg: 7.13 7.12 7.09 8/462 13237
/proc/meminfo: memFree=15724032/32951124 swapFree=57171160/67111528
[pid=13222] ppid=13220 vsize=299632 CPUtime=0.53 cores=1,3,5,7
/proc/13222/stat : 13222 (python) S 13220 13222 12094 0 -1 4202496 10668 933 1 0 43 10 0 0 19 0 3 0 651892040 306823168 7238 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/13222/statm: 74908 7238 1878 364 0 10823 0
[pid=13228] ppid=13222 vsize=288824 CPUtime=0.32 cores=1,3,5,7
/proc/13228/stat : 13228 (python) S 13222 13222 12094 0 -1 4202560 1076 0 0 0 16 16 0 0 19 0 2 0 651892100 295755776 5639 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/13228/statm: 72206 5639 410 364 0 8124 0
[pid=13230] ppid=13228 vsize=1017800 CPUtime=99.33 cores=1,3,5,7
/proc/13230/stat : 13230 (gnovelty+2) T 13228 13230 13230 0 -1 4202496 438 0 0 0 9933 0 0 0 25 0 1 0 651892101 1042227200 177 33554432000 134512640 135663704 4288266848 18446744073709551615 134524857 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/13230/statm: 254450 177 113 282 0 254166 0
[pid=13228/tid=13234] ppid=13222 vsize=288824 CPUtime=0 cores=1,3,5,7
/proc/13228/task/13234/stat : 13234 (python) S 13222 13222 12094 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 651902034 295755776 5639 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 514 18446604463884427712 0 0 -1 1 0 0 0
[pid=13235] ppid=13222 vsize=289388 CPUtime=0 cores=1,3,5,7
/proc/13235/stat : 13235 (python) S 13222 13222 12094 0 -1 4202560 751 0 0 0 0 0 0 0 15 0 1 0 651902037 296333312 5704 33554432000 4194304 5683644 140734586457184 18446744073709551615 270338403910 0 0 16781312 514 13719160737916485896 0 0 17 1 0 0 0
/proc/13235/statm: 72347 5704 348 364 0 8262 0
[pid=13237] ppid=13235 vsize=2824 CPUtime=2.32 cores=1,3,5,7
/proc/13237/stat : 13237 (march_hi) R 13235 13237 13237 0 -1 4202496 735 0 0 0 232 0 0 0 25 0 1 0 651902038 2891776 468 33554432000 4194304 4955518 140733817800800 18446744073709551615 4227565 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/13237/statm: 706 468 104 186 0 519 0
[pid=13222/tid=13229] ppid=13220 vsize=299632 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13229/stat : 13229 (python) S 13220 13222 12094 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 3 0 651892101 306823168 7238 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 7 0 0 0
[pid=13222/tid=13236] ppid=13220 vsize=299632 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13236/stat : 13236 (python) S 13220 13222 12094 0 -1 4202560 23 933 0 0 0 0 0 0 20 0 3 0 651902037 306823168 7238 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 102.5
Current children cumulated vsize (KiB) 1898468

[startup+162.301 s]
/proc/loadavg: 7.04 7.09 7.08 8/462 13239
/proc/meminfo: memFree=15700572/32951124 swapFree=57171160/67111528
[pid=13222] ppid=13220 vsize=299632 CPUtime=0.53 cores=1,3,5,7
/proc/13222/stat : 13222 (python) S 13220 13222 12094 0 -1 4202496 10668 933 1 0 43 10 0 0 19 0 3 0 651892040 306823168 7238 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/13222/statm: 74908 7238 1878 364 0 10823 0
[pid=13228] ppid=13222 vsize=288824 CPUtime=0.32 cores=1,3,5,7

################
# More data... #
################

/proc/13228/stat : 13228 (python) S 13222 13222 12094 0 -1 4202560 1076 0 0 0 16 16 0 0 19 0 2 0 651892100 295755776 5639 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/13228/statm: 72206 5639 410 364 0 8124 0
[pid=13230] ppid=13228 vsize=1017800 CPUtime=99.33 cores=1,3,5,7
/proc/13230/stat : 13230 (gnovelty+2) T 13228 13230 13230 0 -1 4202496 438 0 0 0 9933 0 0 0 25 0 1 0 651892101 1042227200 177 33554432000 134512640 135663704 4288266848 18446744073709551615 134524857 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/13230/statm: 254450 177 113 282 0 254166 0
[pid=13228/tid=13234] ppid=13222 vsize=288824 CPUtime=0 cores=1,3,5,7
/proc/13228/task/13234/stat : 13234 (python) S 13222 13222 12094 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 651902034 295755776 5639 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 514 18446604463884427712 0 0 -1 1 0 0 0
[pid=13235] ppid=13222 vsize=289388 CPUtime=2.63 cores=1,3,5,7
/proc/13235/stat : 13235 (python) S 13222 13222 12094 0 -1 4202560 1148 0 0 0 138 125 0 0 15 0 2 0 651902037 296333312 5757 33554432000 4194304 5683644 140734586457184 18446744073709551615 270338403910 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/13235/statm: 72347 5757 400 364 0 8262 0
[pid=13237] ppid=13235 vsize=90916 CPUtime=693.43 cores=1,3,5,7
/proc/13237/stat : 13237 (march_hi) R 13235 13237 13237 0 -1 4202496 19679 0 0 0 69332 11 0 0 25 0 1 0 651902038 93097984 19412 33554432000 4194304 4955518 140733817800800 18446744073709551615 4311823 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/13237/statm: 22729 19412 104 186 0 22542 0
[pid=13235/tid=13250] ppid=13222 vsize=289388 CPUtime=0 cores=1,3,5,7
/proc/13235/task/13250/stat : 13250 (python) S 13222 13222 12094 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 651941172 296333312 5757 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 514 18446604463884430400 0 0 -1 1 0 0 0
[pid=13251] ppid=13222 vsize=299328 CPUtime=2.58 cores=1,3,5,7
/proc/13251/stat : 13251 (python) S 13222 13222 12094 0 -1 4202560 1146 0 0 0 131 127 0 0 15 0 2 0 651941175 306511872 5681 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/13251/statm: 74832 5681 400 364 0 10744 0
[pid=13253] ppid=13251 vsize=90916 CPUtime=679.37 cores=1,3,5,7
/proc/13253/stat : 13253 (march_hi) T 13251 13253 13253 0 -1 4202496 19228 0 0 0 67926 11 0 0 25 0 1 0 651941176 93097984 18967 33554432000 4194304 4955518 140733447079952 18446744073709551615 4421280 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/13253/statm: 22729 18967 104 186 0 22542 0
[pid=13251/tid=13256] ppid=13222 vsize=299328 CPUtime=0 cores=1,3,5,7
/proc/13251/task/13256/stat : 13256 (python) S 13222 13222 12094 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 651951396 306511872 5681 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 514 18446604459885430016 0 0 -1 1 0 0 0
[pid=13283] ppid=13222 vsize=310124 CPUtime=2.72 cores=1,3,5,7
/proc/13283/stat : 13283 (python) S 13222 13222 12094 0 -1 4202560 1157 0 0 0 141 131 0 0 15 0 2 0 652010285 317566976 5821 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/13283/statm: 77531 5821 400 364 0 13440 0
[pid=13285] ppid=13283 vsize=90916 CPUtime=684.54 cores=1,3,5,7
/proc/13285/stat : 13285 (march_hi) T 13283 13285 13285 0 -1 4202496 19275 0 0 0 68439 15 0 0 25 0 1 0 652010285 93097984 19010 33554432000 4194304 4955518 140736289263344 18446744073709551615 4227824 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/13285/statm: 22729 19010 104 186 0 22542 0
[pid=13283/tid=13289] ppid=13222 vsize=310124 CPUtime=0 cores=1,3,5,7
/proc/13283/task/13289/stat : 13289 (python) S 13222 13222 12094 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 652020514 317566976 5821 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 514 18446604463884428608 0 0 -1 7 0 0 0
[pid=13297] ppid=13222 vsize=320380 CPUtime=0.33 cores=1,3,5,7
/proc/13297/stat : 13297 (python) S 13222 13222 12094 0 -1 4202560 1103 0 0 0 18 15 0 0 19 0 2 0 652049522 328069120 5825 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/13297/statm: 80095 5825 400 364 0 16001 0
[pid=13299] ppid=13297 vsize=1017800 CPUtime=97.47 cores=1,3,5,7
/proc/13299/stat : 13299 (gnovelty+2) T 13297 13299 13299 0 -1 4202496 443 0 0 0 9747 0 0 0 25 0 1 0 652049523 1042227200 177 33554432000 134512640 135663704 4289424800 18446744073709551615 134525678 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/13299/statm: 254450 177 113 282 0 254166 0
[pid=13297/tid=13303] ppid=13222 vsize=320380 CPUtime=0 cores=1,3,5,7
/proc/13297/task/13303/stat : 13303 (python) S 13222 13222 12094 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 652059458 328069120 5825 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 514 18446604465161785728 0 0 -1 5 0 0 0
[pid=13317] ppid=13222 vsize=330636 CPUtime=1.91 cores=1,3,5,7
/proc/13317/stat : 13317 (python) S 13222 13222 12094 0 -1 4202560 1158 0 0 0 98 93 0 0 15 0 2 0 652089349 338571264 5829 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/13317/statm: 82659 5829 400 364 0 18562 0
[pid=13319] ppid=13317 vsize=76216 CPUtime=488.01 cores=1,3,5,7
/proc/13319/stat : 13319 (march_hi) T 13317 13319 13319 0 -1 4202496 14787 0 0 0 48792 9 0 0 25 0 1 0 652089349 78045184 14521 33554432000 4194304 4955518 140736978704400 18446744073709551615 4235288 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/13319/statm: 19054 14521 104 186 0 18867 0
[pid=13317/tid=13323] ppid=13222 vsize=330636 CPUtime=0 cores=1,3,5,7
/proc/13317/task/13323/stat : 13323 (python) S 13222 13222 12094 0 -1 4202560 49 0 0 0 0 0 0 0 15 0 2 0 652099618 338571264 5829 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 514 18446604467451213056 0 0 -1 3 0 0 0
[pid=13339] ppid=13222 vsize=340976 CPUtime=1.49 cores=1,3,5,7
/proc/13339/stat : 13339 (python) S 13222 13222 12094 0 -1 4202560 1175 0 0 0 77 72 0 0 16 0 2 0 652158322 349159424 5846 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/13339/statm: 85244 5846 409 364 0 21144 0
[pid=13341] ppid=13339 vsize=56572 CPUtime=392.86 cores=1,3,5,7
/proc/13341/stat : 13341 (march_hi) T 13339 13341 13341 0 -1 4202496 12381 0 0 0 39277 9 0 0 25 0 1 0 652158323 57929728 12113 33554432000 4194304 4955518 140734026599328 18446744073709551615 4209667 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/13341/statm: 14143 12113 104 186 0 13956 0
[pid=13339/tid=13834] ppid=13222 vsize=340976 CPUtime=0 cores=1,3,5,7
/proc/13339/task/13834/stat : 13834 (python) S 13222 13222 12094 0 -1 4202560 44 0 0 0 0 0 0 0 16 0 2 0 652197797 349159424 5846 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 514 18446604468864431168 0 0 -1 3 0 0 0
[pid=13835] ppid=13222 vsize=351232 CPUtime=1.45 cores=1,3,5,7
/proc/13835/stat : 13835 (python) S 13222 13222 12094 0 -1 4202560 1193 0 0 0 75 70 0 0 15 0 2 0 652197799 359661568 5875 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/13835/statm: 87808 5875 406 364 0 23705 0
[pid=13837] ppid=13835 vsize=56572 CPUtime=392.56 cores=1,3,5,7
/proc/13837/stat : 13837 (march_hi) T 13835 13837 13837 0 -1 4202496 12410 0 0 0 39252 4 0 0 25 0 1 0 652197799 57929728 12139 33554432000 4194304 4955518 140736988403568 18446744073709551615 4261370 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/13837/statm: 14143 12139 104 186 0 13956 0
[pid=13835/tid=13841] ppid=13222 vsize=351232 CPUtime=0 cores=1,3,5,7
/proc/13835/task/13841/stat : 13841 (python) S 13222 13222 12094 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 652207876 359661568 5875 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 514 18446604457378805056 0 0 -1 1 0 0 0
[pid=13222/tid=13229] ppid=13220 vsize=361732 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13229/stat : 13229 (python) S 13220 13222 12094 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 9 0 651892101 370413568 7398 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 7 0 0 0
[pid=13222/tid=13236] ppid=13220 vsize=361732 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13236/stat : 13236 (python) S 13220 13222 12094 0 -1 4202560 30 933 0 0 0 0 0 0 18 0 9 0 651902037 370413568 7398 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 1 0 0 0
[pid=13222/tid=13252] ppid=13220 vsize=361732 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13252/stat : 13252 (python) S 13220 13222 12094 0 -1 4202560 27 933 0 0 0 0 0 0 18 0 9 0 651941175 370413568 7398 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 7 0 0 0
[pid=13222/tid=13284] ppid=13220 vsize=361732 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13284/stat : 13284 (python) S 13220 13222 12094 0 -1 4202560 36 933 0 0 0 0 0 0 18 0 9 0 652010285 370413568 7398 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 7 0 0 0
[pid=13222/tid=13298] ppid=13220 vsize=361732 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13298/stat : 13298 (python) S 13220 13222 12094 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 9 0 652049522 370413568 7398 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 7 0 0 0
[pid=13222/tid=13318] ppid=13220 vsize=361732 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13318/stat : 13318 (python) S 13220 13222 12094 0 -1 4202560 36 933 0 0 0 0 0 0 18 0 9 0 652089349 370413568 7398 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 7 0 0 0
[pid=13222/tid=13340] ppid=13220 vsize=361732 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13340/stat : 13340 (python) S 13220 13222 12094 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 9 0 652158322 370413568 7398 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 3 0 0 0
[pid=13222/tid=13836] ppid=13220 vsize=361732 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13836/stat : 13836 (python) S 13220 13222 12094 0 -1 4202560 29 933 0 0 0 0 0 0 18 0 9 0 652197799 370413568 7398 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3541.98
Current children cumulated vsize (KiB) 5390328

[startup+3559.2 s]
/proc/loadavg: 6.02 6.49 6.83 7/469 13869
/proc/meminfo: memFree=15015820/32951124 swapFree=57171160/67111528
[pid=13222] ppid=13220 vsize=361732 CPUtime=0.98 cores=1,3,5,7
/proc/13222/stat : 13222 (python) S 13220 13222 12094 0 -1 4202496 17035 933 1 0 86 12 0 0 18 0 9 0 651892040 370413568 7398 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/13222/statm: 90433 7398 1890 364 0 26330 0
[pid=13228] ppid=13222 vsize=288824 CPUtime=0.32 cores=1,3,5,7
/proc/13228/stat : 13228 (python) S 13222 13222 12094 0 -1 4202560 1076 0 0 0 16 16 0 0 19 0 2 0 651892100 295755776 5639 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/13228/statm: 72206 5639 410 364 0 8124 0
[pid=13230] ppid=13228 vsize=1017800 CPUtime=99.33 cores=1,3,5,7
/proc/13230/stat : 13230 (gnovelty+2) T 13228 13230 13230 0 -1 4202496 438 0 0 0 9933 0 0 0 25 0 1 0 651892101 1042227200 177 33554432000 134512640 135663704 4288266848 18446744073709551615 134524857 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/13230/statm: 254450 177 113 282 0 254166 0
[pid=13228/tid=13234] ppid=13222 vsize=288824 CPUtime=0 cores=1,3,5,7
/proc/13228/task/13234/stat : 13234 (python) S 13222 13222 12094 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 651902034 295755776 5639 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 514 18446604463884427712 0 0 -1 1 0 0 0
[pid=13235] ppid=13222 vsize=289388 CPUtime=2.64 cores=1,3,5,7
/proc/13235/stat : 13235 (python) S 13222 13222 12094 0 -1 4202560 1150 0 0 0 139 125 0 0 15 0 2 0 651902037 296333312 5757 33554432000 4194304 5683644 140734586457184 18446744073709551615 270338403910 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/13235/statm: 72347 5757 400 364 0 8262 0
[pid=13237] ppid=13235 vsize=90916 CPUtime=693.63 cores=1,3,5,7
/proc/13237/stat : 13237 (march_hi) R 13235 13237 13237 0 -1 4202496 19683 0 0 0 69352 11 0 0 25 0 1 0 651902038 93097984 19416 33554432000 4194304 4955518 140733817800800 18446744073709551615 4289647 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/13237/statm: 22729 19416 104 186 0 22542 0
[pid=13235/tid=13250] ppid=13222 vsize=289388 CPUtime=0 cores=1,3,5,7
/proc/13235/task/13250/stat : 13250 (python) S 13222 13222 12094 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 651941172 296333312 5757 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 514 18446604463884430400 0 0 -1 1 0 0 0
[pid=13251] ppid=13222 vsize=299328 CPUtime=2.58 cores=1,3,5,7
/proc/13251/stat : 13251 (python) S 13222 13222 12094 0 -1 4202560 1146 0 0 0 131 127 0 0 15 0 2 0 651941175 306511872 5681 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/13251/statm: 74832 5681 400 364 0 10744 0
[pid=13253] ppid=13251 vsize=90916 CPUtime=679.37 cores=1,3,5,7
/proc/13253/stat : 13253 (march_hi) T 13251 13253 13253 0 -1 4202496 19228 0 0 0 67926 11 0 0 25 0 1 0 651941176 93097984 18967 33554432000 4194304 4955518 140733447079952 18446744073709551615 4421280 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/13253/statm: 22729 18967 104 186 0 22542 0
[pid=13251/tid=13256] ppid=13222 vsize=299328 CPUtime=0 cores=1,3,5,7
/proc/13251/task/13256/stat : 13256 (python) S 13222 13222 12094 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 651951396 306511872 5681 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 514 18446604459885430016 0 0 -1 1 0 0 0
[pid=13283] ppid=13222 vsize=310124 CPUtime=2.72 cores=1,3,5,7
/proc/13283/stat : 13283 (python) S 13222 13222 12094 0 -1 4202560 1157 0 0 0 141 131 0 0 15 0 2 0 652010285 317566976 5821 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/13283/statm: 77531 5821 400 364 0 13440 0
[pid=13285] ppid=13283 vsize=90916 CPUtime=684.54 cores=1,3,5,7
/proc/13285/stat : 13285 (march_hi) T 13283 13285 13285 0 -1 4202496 19275 0 0 0 68439 15 0 0 25 0 1 0 652010285 93097984 19010 33554432000 4194304 4955518 140736289263344 18446744073709551615 4227824 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/13285/statm: 22729 19010 104 186 0 22542 0
[pid=13283/tid=13289] ppid=13222 vsize=310124 CPUtime=0 cores=1,3,5,7
/proc/13283/task/13289/stat : 13289 (python) S 13222 13222 12094 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 652020514 317566976 5821 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 514 18446604463884428608 0 0 -1 7 0 0 0
[pid=13297] ppid=13222 vsize=320380 CPUtime=0.33 cores=1,3,5,7
/proc/13297/stat : 13297 (python) S 13222 13222 12094 0 -1 4202560 1103 0 0 0 18 15 0 0 19 0 2 0 652049522 328069120 5825 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/13297/statm: 80095 5825 400 364 0 16001 0
[pid=13299] ppid=13297 vsize=1017800 CPUtime=97.47 cores=1,3,5,7
/proc/13299/stat : 13299 (gnovelty+2) T 13297 13299 13299 0 -1 4202496 443 0 0 0 9747 0 0 0 25 0 1 0 652049523 1042227200 177 33554432000 134512640 135663704 4289424800 18446744073709551615 134525678 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/13299/statm: 254450 177 113 282 0 254166 0
[pid=13297/tid=13303] ppid=13222 vsize=320380 CPUtime=0 cores=1,3,5,7
/proc/13297/task/13303/stat : 13303 (python) S 13222 13222 12094 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 652059458 328069120 5825 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 514 18446604465161785728 0 0 -1 5 0 0 0
[pid=13317] ppid=13222 vsize=330636 CPUtime=1.91 cores=1,3,5,7
/proc/13317/stat : 13317 (python) S 13222 13222 12094 0 -1 4202560 1158 0 0 0 98 93 0 0 15 0 2 0 652089349 338571264 5829 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/13317/statm: 82659 5829 400 364 0 18562 0
[pid=13319] ppid=13317 vsize=76216 CPUtime=488.01 cores=1,3,5,7
/proc/13319/stat : 13319 (march_hi) T 13317 13319 13319 0 -1 4202496 14787 0 0 0 48792 9 0 0 25 0 1 0 652089349 78045184 14521 33554432000 4194304 4955518 140736978704400 18446744073709551615 4235288 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/13319/statm: 19054 14521 104 186 0 18867 0
[pid=13317/tid=13323] ppid=13222 vsize=330636 CPUtime=0 cores=1,3,5,7
/proc/13317/task/13323/stat : 13323 (python) S 13222 13222 12094 0 -1 4202560 49 0 0 0 0 0 0 0 15 0 2 0 652099618 338571264 5829 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 514 18446604467451213056 0 0 -1 3 0 0 0
[pid=13339] ppid=13222 vsize=340976 CPUtime=1.49 cores=1,3,5,7
/proc/13339/stat : 13339 (python) S 13222 13222 12094 0 -1 4202560 1175 0 0 0 77 72 0 0 16 0 2 0 652158322 349159424 5846 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/13339/statm: 85244 5846 409 364 0 21144 0
[pid=13341] ppid=13339 vsize=56572 CPUtime=392.86 cores=1,3,5,7
/proc/13341/stat : 13341 (march_hi) T 13339 13341 13341 0 -1 4202496 12381 0 0 0 39277 9 0 0 25 0 1 0 652158323 57929728 12113 33554432000 4194304 4955518 140734026599328 18446744073709551615 4209667 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/13341/statm: 14143 12113 104 186 0 13956 0
[pid=13339/tid=13834] ppid=13222 vsize=340976 CPUtime=0 cores=1,3,5,7
/proc/13339/task/13834/stat : 13834 (python) S 13222 13222 12094 0 -1 4202560 44 0 0 0 0 0 0 0 16 0 2 0 652197797 349159424 5846 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 514 18446604468864431168 0 0 -1 3 0 0 0
[pid=13835] ppid=13222 vsize=351232 CPUtime=1.45 cores=1,3,5,7
/proc/13835/stat : 13835 (python) S 13222 13222 12094 0 -1 4202560 1193 0 0 0 75 70 0 0 15 0 2 0 652197799 359661568 5875 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/13835/statm: 87808 5875 406 364 0 23705 0
[pid=13837] ppid=13835 vsize=56572 CPUtime=392.56 cores=1,3,5,7
/proc/13837/stat : 13837 (march_hi) T 13835 13837 13837 0 -1 4202496 12410 0 0 0 39252 4 0 0 25 0 1 0 652197799 57929728 12139 33554432000 4194304 4955518 140736988403568 18446744073709551615 4261370 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/13837/statm: 14143 12139 104 186 0 13956 0
[pid=13835/tid=13841] ppid=13222 vsize=351232 CPUtime=0 cores=1,3,5,7
/proc/13835/task/13841/stat : 13841 (python) S 13222 13222 12094 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 652207876 359661568 5875 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 514 18446604457378805056 0 0 -1 1 0 0 0
[pid=13222/tid=13229] ppid=13220 vsize=361732 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13229/stat : 13229 (python) S 13220 13222 12094 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 9 0 651892101 370413568 7398 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 7 0 0 0
[pid=13222/tid=13236] ppid=13220 vsize=361732 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13236/stat : 13236 (python) S 13220 13222 12094 0 -1 4202560 30 933 0 0 0 0 0 0 18 0 9 0 651902037 370413568 7398 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 1 0 0 0
[pid=13222/tid=13252] ppid=13220 vsize=361732 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13252/stat : 13252 (python) S 13220 13222 12094 0 -1 4202560 27 933 0 0 0 0 0 0 18 0 9 0 651941175 370413568 7398 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 7 0 0 0
[pid=13222/tid=13284] ppid=13220 vsize=361732 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13284/stat : 13284 (python) S 13220 13222 12094 0 -1 4202560 36 933 0 0 0 0 0 0 18 0 9 0 652010285 370413568 7398 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 7 0 0 0
[pid=13222/tid=13298] ppid=13220 vsize=361732 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13298/stat : 13298 (python) S 13220 13222 12094 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 9 0 652049522 370413568 7398 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 7 0 0 0
[pid=13222/tid=13318] ppid=13220 vsize=361732 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13318/stat : 13318 (python) S 13220 13222 12094 0 -1 4202560 36 933 0 0 0 0 0 0 18 0 9 0 652089349 370413568 7398 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 7 0 0 0
[pid=13222/tid=13340] ppid=13220 vsize=361732 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13340/stat : 13340 (python) S 13220 13222 12094 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 9 0 652158322 370413568 7398 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 3 0 0 0
[pid=13222/tid=13836] ppid=13220 vsize=361732 CPUtime=0 cores=1,3,5,7
/proc/13222/task/13836/stat : 13836 (python) S 13220 13222 12094 0 -1 4202560 29 933 0 0 0 0 0 0 18 0 9 0 652197799 370413568 7398 33554432000 4194304 5683644 140734586457184 18446744073709551615 270350207409 0 0 16781312 2 18446604463969910848 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3542.19
Current children cumulated vsize (KiB) 5390328

Child status: 0
Real time (s): 3559.29
CPU time (s): 3542.45
CPU user time (s): 3535.15
CPU system time (s): 7.29689
CPU usage (%): 99.5268
Max. virtual memory (cumulated for all children) (KiB): 5390332

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3535.15
system time used= 7.29689
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 130134
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 13226
involuntary context switches= 31636

runsolver used 19.598 second user time and 41.3717 second system time

The end

Launcher Data 

Begin job on node143 at 2011-05-17 03:12:44
IDJOB=3386904
IDBENCH=82969
IDSOLVER=1872
FILE ID=node143/3386904-1305594764
RUNJOBID= node143-1305586143-12113
PBS_JOBID= 13350335
Free space on /tmp= 73440 MiB

SOLVER NAME= borg-sat 11.03.02-seq-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v500-c2130-S1205196501-095.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386904-1305594764/watcher-3386904-1305594764 -o /tmp/evaluation-result-3386904-1305594764/solver-3386904-1305594764 -C 5000 -W 5100 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386904-1305594764.cnf -seed 455774198 -budget 5000

TIME LIMIT= 5000 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= b1bf2be38e1cea16efc5e7effa1a10c7
RANDOM SEED=455774198

node143.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      15778828 kB
Buffers:        223224 kB
Cached:        1688312 kB
SwapCached:      39784 kB
Active:        9037956 kB
Inactive:      7838160 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      15778828 kB
SwapTotal:    67111528 kB
SwapFree:     57171160 kB
Dirty:           32140 kB
Writeback:           0 kB
AnonPages:    14924528 kB
Mapped:          36304 kB
Slab:           141196 kB
PageTables:      86848 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 119406072 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73644 MiB
End job on node143 at 2011-05-17 04:12:04