[Version Information]
Phase2 (IST-Java-ICRA) WALi-OpenNWA version: 9042782a9a5ed8987eef2dfeec4e384ee950f24c (2017-05-01 17:41:53 -0500) "Adding benchmarks for model of malware analyzer."
Phase2 (IST-Java-ICRA) duet version: 393b2fa15ae67d4cf44b9a72bd4e0c54307ef649 (2017-05-16 13:07:27 -0400) "Updated the costs for thread.sleep from 10 to 1000"

Phase3 WALi-OpenNWA version: 9042782a9a5ed8987eef2dfeec4e384ee950f24c (2017-05-01 17:41:53 -0500) "Adding benchmarks for model of malware analyzer."
Phase3 duet version: 0fa8064dfa94888565bc782359017482a22e6f46 (2017-05-16 08:41:47 -0400) "Fixed build after merge"
# Installed packages for 4.02.1+PIC:
apron              20160125  APRON numerical abstract domain library
base-bigarray          base  Bigarray library distributed with the OCaml compile
base-bytes             base  Bytes library distributed with the OCaml compiler
base-ocamlbuild        base  OCamlbuild binary and libraries distributed with th
base-threads           base  Threads library distributed with the OCaml compiler
base-unix              base  Unix library distributed with the OCaml compiler
batteries             2.5.3  a community-maintained standard library extension
camlidl                1.05  Stub code generator for OCaml
camlp4               4.02+7  Camlp4 is a system for writing extensible parsers f
cil                20150825  A front-end for the C programming language that fac
conf-gmp                  1  Virtual package relying on a GMP lib system install
conf-m4                   1  Virtual package relying on m4
conf-mathsat              1  Virtual package relying on a MathSAT system install
conf-mpfr                 1  Virtual package relying on library MPFR installatio
conf-perl                 1  Virtual package relying on perl
conf-python-2-7         1.0  Virtual package relying on Python-2.7 installation.
conf-which                1  Virtual package relying on which
cppo                  1.4.1  Equivalent of the C preprocessor for OCaml programs
deriving           20140904  Extension to OCaml for deriving functions from type
mathsat            20161206  MathSAT 5 SMT solver
menhir             20170101  LR(1) parser generator
mlgmpidl              1.2.4  OCaml interface to the GMP library
oasis                 0.4.8  Tooling for building OCaml libraries and applicatio
ocamlbuild                0  Build system distributed with the OCaml compiler si
ocamlfind             1.7.1  A library manager for OCaml
ocamlgraph            1.8.7  A generic graph library for OCaml
ocamlify              0.0.1  Include files in OCaml code
ocamlmod              0.0.8  Generate OCaml modules from source files
OCRS               20170323  Recurrence solver based on operational calculus
optcomp                 1.6  Optional compilation with cpp-like directives
ounit                 2.0.0  Unit testing framework loosely based on HUnit. It i
ppx_deriving            4.1  Type-driven code generation for OCaml >=4.02
ppx_tools        5.0+4.02.0  Tools for authors of ppx rewriters and other syntac
result                  1.2  Compatibility Result module
Z3                 20161217  Z3 SMT solver

Test Name Output No. of Rounds Result Time (Prev.)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/c4b
gcd.c [OCRS][without] 1 FAIL
0.365
kmp.c [OCRS][without] 1 PASS
1.33 (1.169)
qsort.c [OCRS][without] 5 PASS
3.497 (1.987)
speed_pldi09_fig1.c [OCRS][without] 1 PASS
0.359
speed_pldi09_fig4_2.c [OCRS][without] 1 FAIL
0.387
speed_pldi09_fig4_4.c [OCRS][without] 1 PASS
0.438
speed_pldi09_fig4_5.c [OCRS][without] 1 PASS
0.354
speed_pldi10_ex1.c [OCRS][without] 1 PASS
3.812 (0.803)
speed_pldi10_ex3.c [OCRS][without] 1 PASS
0.447
speed_pldi10_ex4.c [OCRS][without] EXCEPTION
0.407
speed_popl10_fig2_1.c [OCRS][without] 1 PASS
0.39
speed_popl10_fig2_2.c [OCRS][without] 1 FAIL
0.383
speed_popl10_nested_multiple.c [OCRS][without] 1 PASS
0.43
speed_popl10_nested_single.c [OCRS][without] 1 PASS
0.407
speed_popl10_sequential_single.c [OCRS][without] 1 PASS
0.363
speed_popl10_simple_multiple.c [OCRS][without] 1 PASS
0.383
speed_popl10_simple_single_2.c [OCRS][without] 1 PASS
0.413
speed_popl10_simple_single.c [OCRS][without] 1 PASS
0.331
t07.c [OCRS][without] 1 PASS
0.382
t08.c [OCRS][without] 1 PASS
0.344
t09.c [OCRS][without] 1 PASS
0.347
t10.c [OCRS][without] 1 PASS
0.377
t11.c [OCRS][without] 1 PASS
0.379
t13.c [OCRS][without] 1 FAIL
0.38
t15.c [OCRS][without] 1 PASS
0.37
t16.c [OCRS][without] 1 PASS
0.357
t19.c [OCRS][without] 1 PASS
0.353
t20.c [OCRS][without] 1 PASS
0.342
t27.c [OCRS][without] 1 PASS
0.432
t28.c [OCRS][without] 1 PASS
0.392
t30.c [OCRS][without] 1 FAIL
0.368
t37.c [OCRS][without] 2 PASS
0.491
t39.c [OCRS][without] 2 PASS
0.501
t46.c [OCRS][without] 1 PASS
0.346
t47.c [OCRS][without] 1 PASS
0.36
Total Below Time = 21.017 (was 16.221)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/misc-recursive
mult-rec.c [OCRS][without] 2 PASS
PASS
0.394
qsort_full.c [OCRS][without] 5 TIMEOUT
300.005 (41.793)
rec1-param_safe.c [OCRS][without] 2 PASS
PASS
PASS
0.347
rec1-param_unsafe.c [OCRS][without] 2 PASS
0.342
rec1_safe.c [OCRS][without] 2 PASS
PASS
PASS
0.334
rec1_unsafe.c [OCRS][without] 2 PASS
0.363
rec2-param_safe.c [OCRS][without] 2 PASS
PASS
PASS
0.355
rec2-param_unsafe.c [OCRS][without] 2 PASS
0.341
rec2_safe.c [OCRS][without] 2 PASS
PASS
PASS
0.34
rec2_unsafe.c [OCRS][without] 2 PASS
0.337
rec-test.c [OCRS][without] 2 PASS
0.331
sas03_bothbranches.c [OCRS][without] 7 PASS
1.283
sas03.c [OCRS][without] 2 PASS
0.805
simulated_lese_recursive.c [OCRS][without] 2 PASS
0.391
Total Below Time = 305.968 (was 47.742)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/duet
count_up_down_safe.c [OCRS][without] 1 PASS
0.309
count_up_down_unsafe.c [OCRS][without] 1 PASS
0.334
divide.c [OCRS][without] 1 PASS
0.324
factor_unsafe.c [OCRS][without] 1 PASS
0.316
for_infinite_loop_1_safe.c [OCRS][without] 1 PASS
0.325
for_infinite_loop_2_safe.c [OCRS][without] 1 PASS
0.322
interproc.c [OCRS][without] 1 PASS
0.315
mult.c [OCRS][without] 1 PASS
PASS
0.334
pointer_arith.c [OCRS][without] 1 PASS
PASS
PASS
PASS
0.314
quotient.c [OCRS][without] 3 PASS
0.419
subtract.c [OCRS][without] 1 PASS
0.326
sum01_bug02_sum01_bug02_base.case_unsafe.c [OCRS][without] 1 PASS
0.35
sum01_bug02_unsafe.c [OCRS][without] 1 PASS
0.389
sum01_real_safe.c [OCRS][without] 1 PASS
0.32
sum01_safe.c [OCRS][without] 1 PASS
0.327
sum01_unsafe.c [OCRS][without] 1 PASS
0.355
sum02_safe.c [OCRS][without] 1 PASS
0.336
sum03_safe.c [OCRS][without] 1 PASS
0.327
sum03_unsafe.c [OCRS][without] 1 PASS
0.39
sum04_safe.c [OCRS][without] 1 PASS
0.321
sum04_unsafe.c [OCRS][without] 1 PASS
0.361
terminator_02_safe.c [OCRS][without] 1 PASS
0.336
terminator_02_unsafe.c [OCRS][without] 1 PASS
0.359
terminator_03_safe.c [OCRS][without] 1 PASS
0.326
terminator_03_unsafe.c [OCRS][without] 1 PASS
0.329
token_ring01_safe.c [OCRS][without] TIMEOUT
300.007
token_ring01_unsafe.c [OCRS][without] TIMEOUT
300.007
toy_safe.c [OCRS][without] EXCEPTION
173.7
trex01_safe.c [OCRS][without] 1 PASS
0.532
trex01_unsafe.c [OCRS][without] 1 PASS
0.356 (0.397)
trex02_safe2.c [OCRS][without] 3 PASS
0.392
trex02_safe.c [OCRS][without] 3 PASS
0.384
trex02_unsafe.c [OCRS][without] 3 PASS
0.384
trex03_safe.c [OCRS][without] 1 PASS
0.46
trex03_unsafe.c [OCRS][without] 1 PASS
0.454
trex04_safe.c [OCRS][without] 1 PASS
0.344
while_infinite_loop_1_safe.c [OCRS][without] 1 PASS
0.3
while_infinite_loop_2_safe.c [OCRS][without] 1 PASS
0.307
while_infinite_loop_3_safe.c [OCRS][without] 1 PASS
0.308
Total Below Time = 786.399 (was 789.88)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests
above_litmus.c [OCRS][without] 3 PASS
0.429
blogger_simplified1.c [OCRS][without] 3 PASS
0.997
divided_difference2.c [OCRS][without] TIMEOUT
300.005
mult-proc.c [OCRS][without] 3 PASS
PASS
0.36
mult-proc-params.c [OCRS][without] 3 PASS
PASS
0.387
popall.c [OCRS][without] 1 PASS
PASS
PASS
PASS
PASS
0.424
simulated_scc.c [OCRS][without] 1 PASS
PASS
0.517
Total Below Time = 303.119 (was 303.028)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/STAC/polynomial/assert
degree1_monomial.c [OCRS][without] 1 PASS
0.323
degree2_binomial.c [OCRS][without] 1 PASS
0.456
degree2_monomial.c [OCRS][without] 1 PASS
0.39
degree3_binomial.c [OCRS][without] 1 PASS
0.913 (0.802)
degree3_monomial.c [OCRS][without] 1 PASS
0.413
degree4_binomial.c [OCRS][without] 1 PASS
1.127
degree4_monomial.c [OCRS][without] 1 PASS
0.485
degree5_binomial.c [OCRS][without] 1 PASS
1.736
degree5_monomial.c [OCRS][without] 1 PASS
0.565
Total Below Time = 6.408 (was 6.386)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/snlee/snlee_tests
cubic_with_square.c [OCRS][without] 1 PASS
0.342
cubic_with_square_interproc.c [OCRS][without] 2 PASS
0.377
cubic_with_square_nonlinear.c [OCRS][without] 1 PASS
0.366
cubic_with_square_nonlinear_interproc.c [OCRS][without] 2 PASS
0.379
cubic_with_square_nonlinear_unsafe.c [OCRS][without] 1 PASS
0.354
cubic_with_square_unsafe.c [OCRS][without] 1 PASS
0.366
multi-input.c [OCRS][without] 1 PASS
0.402
multi-input_unsafe.c [OCRS][without] 1 PASS
0.444
nondet_loop_bound.c [OCRS][without] 1 PASS
0.362
nondet_loop_bound_quartic.c [OCRS][without] 1 PASS
0.362
nondet_loop_bound_quartic_unsafe.c [OCRS][without] 1 PASS
0.361
nondet_loop_bound_unsafe.c [OCRS][without] 1 PASS
0.366
nonlinear_stratified.c [OCRS][without] 1 FAIL
0.347
nonlinear_stratified_unsafe.c [OCRS][without] 1 PASS
0.359
quartic_with_cube.c [OCRS][without] 1 PASS
0.339
quartic_with_cube_nonlinear.c [OCRS][without] 1 PASS
0.354
quartic_with_cube_nonlinear_unsafe.c [OCRS][without] 1 PASS
0.387
quartic_with_cube_unsafe.c [OCRS][without] 1 PASS
0.366
quintic_with_quartic.c [OCRS][without] 1 PASS
0.384
quintic_with_quartic_nonlinear.c [OCRS][without] 1 PASS
0.4
quintic_with_quartic_nonlinear_unsafe.c [OCRS][without] 1 PASS
0.375 (0.461)
quintic_with_quartic_unsafe.c [OCRS][without] 1 PASS
0.354
Total Below Time = 8.146 (was 8.193)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/STAC/FiniteDifferencing
degree2.c [OCRS][without] 1 PASS
0.501
degree2_FD_AllAux_AllAssm.c [OCRS][without] 1 PASS
3.584 (3.035)
degree2_FD_AllAux_FewAssm.c [OCRS][without] 1 PASS
5.449 (3.036)
degree2_FD_FewAux_FewAssm.c [OCRS][without] 1 PASS
0.698
degree2_FD_FewAux_FewAssm_NonLinAsrt.c [OCRS][without] 1 PASS
0.696
degree3.c [OCRS][without] 1 PASS
0.546
degree3_FD.c [OCRS][without] 1 PASS
0.739
degree4.c [OCRS][without] 1 PASS
0.395
Total Below Time = 12.608 (was 9.656)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/STAC/LESE
loop.c [OCRS][without] 1 PASS
0.398
loop_unsafe.c [OCRS][without] 1 PASS
0.405
simulated_lese_parser.c [OCRS][without] 1 PASS
0.322
simulated_lese_sentinel.c [OCRS][without] 1 PASS
0.337
Total Below Time = 1.462 (was 1.483)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/STAC/LowerBound
lower_bound.c [OCRS][without] 1 PASS
PASS
PASS
0.337
Total Below Time = 0.337 (was 0.332)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/STAC/LZ
lz77.c [OCRS][without] TIMEOUT
300.016
Total Below Time = 300.016 (was 300.007)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/sv-benchmarks/loop-acceleration
array_false-unreach-call1.c [OCRS][without] 1 PASS
0.342
array_false-unreach-call2.c [OCRS][without] 1 PASS
0.358
array_false-unreach-call3.c [OCRS][without] 1 PASS
0.338
array_true-unreach-call1.c [OCRS][without] 1 FAIL
0.357
array_true-unreach-call2.c [OCRS][without] 1 FAIL
0.362
array_true-unreach-call3.c [OCRS][without] 1 PASS
0.348
array_true-unreach-call4.c [OCRS][without] 1 FAIL
0.361
const_false-unreach-call1.c [OCRS][without] 1 PASS
0.32
const_true-unreach-call1.c [OCRS][without] 1 PASS
0.316
diamond_false-unreach-call1.c [OCRS][without] 1 PASS
0.343
diamond_true-unreach-call1.c [OCRS][without] 1 PASS
0.331
diamond_true-unreach-call2.c [OCRS][without] 1 PASS
0.376
functions_false-unreach-call1.c [OCRS][without] 3 PASS
0.372
functions_true-unreach-call1.c [OCRS][without] 3 PASS
0.342
multivar_false-unreach-call1.c [OCRS][without] 1 PASS
0.326
multivar_true-unreach-call1.c [OCRS][without] 1 PASS
0.334
nested_false-unreach-call1.c [OCRS][without] 1 PASS
0.332
nested_true-unreach-call1.c [OCRS][without] 1 PASS
0.338
overflow_true-unreach-call1.c [OCRS][without] 1 PASS
0.319
phases_false-unreach-call1.c [OCRS][without] 1 PASS
0.338
phases_false-unreach-call2.c [OCRS][without] 1 PASS
0.342
phases_true-unreach-call1.c [OCRS][without] 1 PASS
0.345
phases_true-unreach-call2.c [OCRS][without] 1 PASS
0.345
simple_false-unreach-call1.c [OCRS][without] 1 PASS
0.312
simple_false-unreach-call2.c [OCRS][without] 1 PASS
0.32
simple_false-unreach-call3.c [OCRS][without] 1 PASS
0.319
simple_false-unreach-call4.c [OCRS][without] 1 PASS
0.309
simple_true-unreach-call1.c [OCRS][without] 1 PASS
0.306
simple_true-unreach-call2.c [OCRS][without] 1 PASS
0.316
simple_true-unreach-call3.c [OCRS][without] 1 PASS
0.327
simple_true-unreach-call4.c [OCRS][without] 1 PASS
0.324
underapprox_false-unreach-call1.c [OCRS][without] 1 PASS
0.33
underapprox_false-unreach-call2.c [OCRS][without] 1 PASS
0.341
underapprox_true-unreach-call1.c [OCRS][without] 1 FAIL
0.319
underapprox_true-unreach-call2.c [OCRS][without] 1 PASS
0.334
Total Below Time = 11.742 (was 11.543)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/sv-benchmarks/loop-invgen
apache-escape-absolute_true-unreach-call.c [OCRS][without] 1 PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
0.833
apache-get-tag_true-unreach-call.c [OCRS][without] 1 PASS
PASS
PASS
FAIL
PASS
FAIL
PASS
PASS
PASS
FAIL
PASS
PASS
PASS
FAIL
PASS
FAIL
0.475
down_true-unreach-call.c [OCRS][without] 1 PASS
0.325
fragtest_simple_true-unreach-call.c [OCRS][without] 1 PASS
0.378
half_2_true-unreach-call.c [OCRS][without] 1 PASS
0.352
heapsort_true-unreach-call.c [OCRS][without] 1 TIMEOUT
300.152 (2.316)
id_build_true-unreach-call.c [OCRS][without] 1 PASS
PASS
0.33
id_trans_false-unreach-call.c [OCRS][without] 1 FAIL
FAIL
FAIL
PASS
0.337
large_const_true-unreach-call.c [OCRS][without] 1 PASS
0.418
MADWiFi-encode_ie_ok_true-unreach-call.c [OCRS][without] 1 PASS
PASS
0.365
nested6_true-unreach-call.c [OCRS][without] 1 FAIL
FAIL
PASS
0.417
nested9_true-unreach-call.c [OCRS][without] 1 PASS
0.44
nest-if3_true-unreach-call.c [OCRS][without] 1 PASS
0.393
NetBSD_loop_true-unreach-call.c [OCRS][without] 1 PASS
PASS
0.337
sendmail-close-angle_true-unreach-call.c [OCRS][without] 1 PASS
FAIL
PASS
FAIL
PASS
FAIL
PASS
FAIL
0.37
seq_true-unreach-call.c [OCRS][without] 1 PASS
0.378
SpamAssassin-loop_true-unreach-call.c [OCRS][without] TIMEOUT
300.109 (1.199)
string_concat-noarr_true-unreach-call.c [OCRS][without] 1 PASS
0.348
up_true-unreach-call.c [OCRS][without] 1 PASS
0.34
Total Below Time = 607.097 (was 10.286)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/sv-benchmarks/loop-lit
afnp2014_true-unreach-call.c [OCRS][without] 1 PASS
0.326
bhmr2007_true-unreach-call.c [OCRS][without] 1 PASS
0.347
cggmp2005b_true-unreach-call.c [OCRS][without] 1 PASS
0.399 (0.361)
cggmp2005_true-unreach-call.c [OCRS][without] 1 PASS
0.331
cggmp2005_variant_true-unreach-call.c [OCRS][without] 1 PASS
0.328
css2003_true-unreach-call.c [OCRS][without] 1 PASS
0.483
ddlm2013_true-unreach-call.c [OCRS][without] 1 PASS
0.375
gcnr2008_false-unreach-call.c [OCRS][without] 1 TIMEOUT
300.177 (0.854)
gj2007b_true-unreach-call.c [OCRS][without] 1 PASS
FAIL
0.361
gj2007_true-unreach-call.c [OCRS][without] 1 PASS
0.341
gr2006_true-unreach-call.c [OCRS][without] 1 PASS
0.338
gsv2008_true-unreach-call.c [OCRS][without] 1 PASS
0.325
hhk2008_true-unreach-call.c [OCRS][without] 1 PASS
0.331
jm2006_true-unreach-call.c [OCRS][without] 1 PASS
0.365
jm2006_variant_true-unreach-call.c [OCRS][without] 1 PASS
0.396 (0.349)
mcmillan2006_true-unreach-call.c [OCRS][without] 1 FAIL
0.387
Total Below Time = 305.61 (was 6.12)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/sv-benchmarks/loop-new
count_by_1_true-unreach-call.c [OCRS][without] 1 PASS
0.315
count_by_1_variant_true-unreach-call.c [OCRS][without] 1 PASS
0.332
count_by_2_true-unreach-call.c [OCRS][without] 1 PASS
0.321
count_by_k_true-unreach-call.c [OCRS][without] 1 PASS
0.329
count_by_nondet_true-unreach-call.c [OCRS][without] 1 FAIL
0.329
gauss_sum_true-unreach-call.c [OCRS][without] 1 PASS
0.324
half_true-unreach-call.c [OCRS][without] EXCEPTION
0.341
nested_true-unreach-call.c [OCRS][without] 1 PASS
0.382
Total Below Time = 2.673 (was 2.716)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/sv-benchmarks/loops
array_false-unreach-call.c [OCRS][without] 1 PASS
0.407
array_true-unreach-call.c [OCRS][without] 1 FAIL
0.435
bubble_sort_false-unreach-call.c [OCRS][without] 4 PASS
PASS
142.978
bubble_sort_true-unreach-call.c [OCRS][without] 1 3.982
compact_false-unreach-call.c [OCRS][without] 1 PASS
0.365
count_up_down_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.331
count_up_down_true-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.33
eureka_01_false-unreach-call.c [OCRS][without] 1 PASS
2.194
eureka_01_true-unreach-call.c [OCRS][without] 1 FAIL
1.962
eureka_05_true-unreach-call.c [OCRS][without] 1 FAIL
1.085 (0.722)
for_bounded_loop1_false-unreach-call_true-termination.c [OCRS][without] 1 FAIL
FAIL
PASS
0.322
for_infinite_loop_1_true-unreach-call_false-termination.c [OCRS][without] 1 PASS
0.328
for_infinite_loop_2_true-unreach-call_false-termination.c [OCRS][without] 1 PASS
0.316
heavy_false-unreach-call.c [OCRS][without] 1 PASS
0.523
heavy_true-unreach-call.c [OCRS][without] 1 PASS
0.506
insertion_sort_false-unreach-call.c [OCRS][without] 1 PASS
2.206
insertion_sort_true-unreach-call.c [OCRS][without] 1 FAIL
0.734
invert_string_false-unreach-call.c [OCRS][without] 1 PASS
0.507
invert_string_true-unreach-call.c [OCRS][without] 1 FAIL
0.505
linear_sea.ch_true-unreach-call.c [OCRS][without] 1 FAIL
0.385
linear_search_false-unreach-call.c [OCRS][without] 1 EXCEPTION
0.401
lu.cmp_true-unreach-call.c [OCRS][without] TIMEOUT
300.264 (122.729)
ludcmp_false-unreach-call.c [OCRS][without] TIMEOUT
300.248 (121.286)
matrix_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.963 (0.844)
matrix_true-unreach-call_true-termination.c [OCRS][without] 1 FAIL
0.513
n.c11_true-unreach-call.c [OCRS][without] 3 FAIL
0.469
n.c24_false-unreach-call.c [OCRS][without] 3 PASS
22.965
n.c40_true-unreach-call.c [OCRS][without] 1 FAIL
0.376
nec11_false-unreach-call.c [OCRS][without] 1 PASS
0.373
nec20_false-unreach-call.c [OCRS][without] 1 PASS
0.372
nec40_true-unreach-call.c [OCRS][without] 1 FAIL
0.378
string_false-unreach-call.c [OCRS][without] 1 PASS
0.675
string_true-unreach-call.c [OCRS][without] 1 FAIL
0.642
sum01_bug02_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.394
sum01_bug02_sum01_bug02_base.case_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.338
sum01_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.361
sum01_true-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.329
sum03_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.473
sum03_true-unreach-call_false-termination.c [OCRS][without] 1 PASS
0.356
sum04_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.367
sum04_true-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.323
sum_array_false-unreach-call.c [OCRS][without] 1 PASS
0.662
sum_array_true-unreach-call.c [OCRS][without] 1 FAIL
0.722
terminator_01_false-unreach-call_false-termination.c [OCRS][without] 1 PASS
0.344
terminator_02_false-unreach-call_true-termination.c [OCRS][without] 3 PASS
0.429
terminator_02_true-unreach-call_true-termination.c [OCRS][without] 2 PASS
0.356
terminator_03_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.324
terminator_03_true-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.32
trex01_false-unreach-call_true-termination.c [OCRS][without] 3 PASS
0.462
trex01_true-unreach-call.c [OCRS][without] 3 PASS
0.785 (0.651)
trex02_false-unreach-call_true-termination.c [OCRS][without] 3 PASS
0.401
trex02_true-unreach-call_true-termination.c [OCRS][without] 3 PASS
0.387
trex03_false-unreach-call_true-termination.c [OCRS][without] 3 PASS
0.867
trex03_true-unreach-call.c [OCRS][without] 3 PASS
0.821
trex04_true-unreach-call_false-termination.c [OCRS][without] 1 PASS
0.38
veris.c_NetBSD-libc__loop_true-unreach-call.c [OCRS][without] 1 PASS
0.329
veris.c_OpenSER__cases1_stripFullBoth_arr_true-unreach-call.c [OCRS][without] 3 23.714
veris.c_sendmail__tTflag_arr_one_loop_true-unreach-call.c [OCRS][without] 1 PASS
0.35
verisec_NetBSD-libc__loop_false-unreach-call.c [OCRS][without] 1 PASS
0.392 (0.351)
verisec_OpenSER__cases1_stripFullBoth_arr_false-unreach-call.c [OCRS][without] 3 PASS
17.126
vogal_false-unreach-call.c [OCRS][without] 1 PASS
0.846 (0.755)
vogal_true-unreach-call.c [OCRS][without] 1 FAIL
0.863 (0.748)
while_infinite_loop_1_true-unreach-call_false-termination.c [OCRS][without] 1 PASS
0.317
while_infinite_loop_2_true-unreach-call_false-termination.c [OCRS][without] 1 PASS
0.303
while_infinite_loop_3_true-unreach-call_false-termination.c [OCRS][without] 1 PASS
0.301
while_infinite_loop_4_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.327
Total Below Time = 844.109 (was 480.553)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/sv-benchmarks/recursive
Ackermann01_true-unreach-call.c [OCRS][without] TIMEOUT
300.021 (2.425)
Ackermann02_false-unreach-call_false-termination.c [OCRS][without] TIMEOUT
300.021 (2.334)
Ackermann03_true-unreach-call.c [OCRS][without] TIMEOUT
300.02 (2.399)
Ackermann04_true-unreach-call.c [OCRS][without] TIMEOUT
300.019 (2.451)
Addition01_true-unreach-call_true-termination.c [OCRS][without] 2 PASS
0.569
Addition02_false-unreach-call_false-termination.c [OCRS][without] 2 PASS
0.586
Addition03_false-no-overflow.c [OCRS][without] 2 PASS
0.57
Addition03_true-unreach-call.c [OCRS][without] 2 PASS
0.566
BallRajamani-SPIN2000-Fig1_false-unreach-call.c [OCRS][without] 2 PASS
0.444
EvenOdd01_true-unreach-call_true-termination.c [OCRS][without] 2 PASS
0.411
EvenOdd03_false-unreach-call_false-termination.c [OCRS][without] 2 PASS
0.416
Fibonacci01_true-unreach-call.c [OCRS][without] EXCEPTION
0.966 (300.01)
Fibonacci02_true-unreach-call_true-termination.c [OCRS][without] EXCEPTION
0.93 (300.007)
Fibonacci03_true-unreach-call_true-termination.c [OCRS][without] EXCEPTION
0.927 (300.009)
Fibonacci04_false-unreach-call_true-termination.c [OCRS][without] EXCEPTION
0.924 (300.011)
Fibonacci05_false-unreach-call_true-termination.c [OCRS][without] EXCEPTION
0.94 (300.007)
gcd01_true-unreach-call_true-termination.c [OCRS][without] 2 PASS
0.573
gcd02_true-unreach-call.c [OCRS][without] 2 FAIL
PASS
1.346 (1.209)
McCarthy91_false-unreach-call_false-termination.c [OCRS][without] 7 PASS
1.452 (1.128)
McCarthy91_true-unreach-call.c [OCRS][without] 7 FAIL
1.396 (1.119)
MultCommutative_true-unreach-call_true-termination.c [OCRS][without] 2 FAIL
0.667 (0.588)
Primes_true-unreach-call.c [OCRS][without] 3 FAIL
2.137 (1.938)
recHanoi01_true-unreach-call_true-termination.c [OCRS][without] 9 FAIL
5.736
recHanoi02_true-unreach-call_true-termination.c [OCRS][without] 2 FAIL
0.381
recHanoi03_true-unreach-call_true-termination.c [OCRS][without] 2 FAIL
0.381
Total Below Time = 1222.399 (was 1526.392)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/sv-benchmarks/recursive-simple
afterrec_2calls_false-unreach-call.c [OCRS][without] 2 PASS
PASS
0.425
afterrec_2calls_true-unreach-call.c [OCRS][without] 2 PASS
PASS
0.374
afterrec_false-unreach-call.c [OCRS][without] 2 PASS
0.35
afterrec_true-unreach-call.c [OCRS][without] 2 PASS
0.357
fibo_10_false-unreach-call.c [OCRS][without] EXCEPTION
0.964 (300.008)
fibo_10_true-unreach-call.c [OCRS][without] EXCEPTION
0.991 (300.01)
fibo_15_false-unreach-call.c [OCRS][without] EXCEPTION
0.953 (300.007)
fibo_15_true-unreach-call.c [OCRS][without] EXCEPTION
0.946 (300.007)
fibo_20_false-unreach-call.c [OCRS][without] EXCEPTION
0.967 (300.01)
fibo_20_true-unreach-call.c [OCRS][without] EXCEPTION
0.955 (300.007)
fibo_25_false-unreach-call.c [OCRS][without] EXCEPTION
0.927 (300.009)
fibo_25_true-unreach-call.c [OCRS][without] EXCEPTION
0.921 (300.011)
fibo_2calls_10_false-unreach-call.c [OCRS][without] EXCEPTION
17.239 (130.262)
fibo_2calls_10_true-unreach-call.c [OCRS][without] EXCEPTION
17.373 (129.404)
fibo_2calls_15_false-unreach-call.c [OCRS][without] EXCEPTION
17.373 (131.358)
fibo_2calls_15_true-unreach-call.c [OCRS][without] EXCEPTION
17.2 (129.957)
fibo_2calls_20_false-unreach-call.c [OCRS][without] EXCEPTION
17.243 (128.648)
fibo_2calls_20_true-unreach-call.c [OCRS][without] EXCEPTION
17.211 (130.779)
fibo_2calls_25_false-unreach-call.c [OCRS][without] EXCEPTION
17.395 (130.701)
fibo_2calls_25_true-unreach-call.c [OCRS][without] EXCEPTION
17.469 (128.173)
fibo_2calls_2_false-unreach-call.c [OCRS][without] EXCEPTION
17.444 (129.407)
fibo_2calls_2_true-unreach-call.c [OCRS][without] EXCEPTION
17.027 (130.714)
fibo_2calls_4_false-unreach-call.c [OCRS][without] EXCEPTION
17.376 (129.796)
fibo_2calls_4_true-unreach-call.c [OCRS][without] EXCEPTION
17.115 (130.334)
fibo_2calls_5_false-unreach-call.c [OCRS][without] EXCEPTION
17.261 (128.011)
fibo_2calls_5_true-unreach-call.c [OCRS][without] EXCEPTION
17.318 (131.778)
fibo_2calls_6_false-unreach-call.c [OCRS][without] EXCEPTION
17.412 (130.777)
fibo_2calls_6_true-unreach-call.c [OCRS][without] EXCEPTION
17.431 (127.911)
fibo_2calls_8_false-unreach-call.c [OCRS][without] EXCEPTION
17.177 (126.874)
fibo_2calls_8_true-unreach-call.c [OCRS][without] EXCEPTION
17.173 (127.764)
fibo_5_false-unreach-call.c [OCRS][without] EXCEPTION
0.946 (300.009)
fibo_5_true-unreach-call.c [OCRS][without] EXCEPTION
0.924 (300.01)
fibo_7_false-unreach-call.c [OCRS][without] EXCEPTION
0.939 (300.007)
fibo_7_true-unreach-call.c [OCRS][without] EXCEPTION
0.915 (300.007)
id2_b2_o3_true-unreach-call.c [OCRS][without] 2 PASS
0.936
id2_b3_o2_false-unreach-call.c [OCRS][without] 2 PASS
1.0
id2_b3_o5_true-unreach-call.c [OCRS][without] 2 PASS
0.955
id2_b5_o10_true-unreach-call.c [OCRS][without] 2 PASS
0.977
id2_i5_o5_false-unreach-call.c [OCRS][without] 2 PASS
0.421
id2_i5_o5_true-unreach-call.c [OCRS][without] 2 PASS
0.398
id_b2_o3_true-unreach-call.c [OCRS][without] 2 PASS
0.588
id_b3_o2_false-unreach-call.c [OCRS][without] 2 PASS
0.573
id_b3_o5_true-unreach-call.c [OCRS][without] 2 PASS
0.585
id_b5_o10_true-unreach-call.c [OCRS][without] 2 PASS
0.596
id_i10_o10_false-unreach-call.c [OCRS][without] 2 PASS
0.378
id_i10_o10_true-unreach-call.c [OCRS][without] 2 PASS
0.367
id_i15_o15_false-unreach-call.c [OCRS][without] 2 PASS
0.374
id_i15_o15_true-unreach-call.c [OCRS][without] 2 PASS
0.37
id_i20_o20_false-unreach-call.c [OCRS][without] 2 PASS
0.376
id_i20_o20_true-unreach-call.c [OCRS][without] 2 PASS
0.37
id_i25_o25_false-unreach-call.c [OCRS][without] 2 PASS
0.367
id_i25_o25_true-unreach-call.c [OCRS][without] 2 PASS
0.39
id_i5_o5_false-unreach-call.c [OCRS][without] 2 PASS
0.373
id_i5_o5_true-unreach-call.c [OCRS][without] 2 PASS
0.381
id_o1000_false-unreach-call.c [OCRS][without] 2 PASS
0.367
id_o100_false-unreach-call.c [OCRS][without] 2 PASS
0.375
id_o10_false-unreach-call.c [OCRS][without] 2 PASS
0.381
id_o200_false-unreach-call.c [OCRS][without] 2 PASS
0.381
id_o20_false-unreach-call.c [OCRS][without] 2 PASS
0.365
id_o3_false-unreach-call.c [OCRS][without] 2 PASS
0.375
sum_10x0_false-unreach-call.c [OCRS][without] 2 PASS
0.39
sum_10x0_true-unreach-call.c [OCRS][without] 2 PASS
0.386
sum_15x0_false-unreach-call.c [OCRS][without] 2 PASS
0.386
sum_15x0_true-unreach-call.c [OCRS][without] 2 PASS
0.381
sum_20x0_false-unreach-call.c [OCRS][without] 2 PASS
0.403
sum_20x0_true-unreach-call.c [OCRS][without] 2 PASS
0.392
sum_25x0_false-unreach-call.c [OCRS][without] 2 PASS
0.395
sum_25x0_true-unreach-call.c [OCRS][without] 2 PASS
0.375
sum_2x3_false-unreach-call.c [OCRS][without] 2 PASS
0.395
sum_2x3_true-unreach-call.c [OCRS][without] 2 PASS
0.395
sum_non_eq_false-unreach-call.c [OCRS][without] 2 PASS
0.429
sum_non_eq_true-unreach-call.c [OCRS][without] 2 PASS
0.418
sum_non_false-unreach-call.c [OCRS][without] 2 PASS
0.389
sum_non_true-unreach-call.c [OCRS][without] 2 PASS
0.383
Total Below Time = 342.627 (was 5952.267)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/rec-sv-benchmarks/rec-loop-lit
rec-afnp2014_true-unreach-call.c [OCRS][without] 2 PASS
0.381
rec-bhmr2007_true-unreach-call.c [OCRS][without] 2 PASS
0.416
rec-cggmp2005b_true-unreach-call.c [OCRS][without] 3 PASS
0.742
rec-cggmp2005_true-unreach-call.c [OCRS][without] 2 PASS
0.336
rec-cggmp2005_variant_true-unreach-call.c [OCRS][without] 2 PASS
0.371
rec-css2003_true-unreach-call.c [OCRS][without] 2 PASS
0.606
rec-ddlm2013_true-unreach-call.c [OCRS][without] EXCEPTION
0.355 (0.42)
rec-gcnr2008_false-unreach-call.c [OCRS][without] 2 PASS
0.878
rec-gj2007b_true-unreach-call.c [OCRS][without] 2 FAIL
FAIL
0.378
rec-gj2007_true-unreach-call.c [OCRS][without] 2 FAIL
0.375
rec-gr2006_true-unreach-call.c [OCRS][without] 2 FAIL
0.398
rec-gsv2008_true-unreach-call.c [OCRS][without] 2 PASS
0.345
rec-hhk2008_true-unreach-call.c [OCRS][without] 2 PASS
0.354
rec-jm2006_true-unreach-call.c [OCRS][without] 2 PASS
0.366
rec-jm2006_variant_true-unreach-call.c [OCRS][without] 2 PASS
0.373
rec-mcmillan2006_true-unreach-call.c [OCRS][without] 2 FAIL
0.425
Total Below Time = 7.099 (was 7.026)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/rec-sv-benchmarks/rec-loop-new
rec-count_by_1_true-unreach-call.c [OCRS][without] 2 PASS
0.328
rec-count_by_1_variant_true-unreach-call.c [OCRS][without] 2 PASS
0.344
rec-count_by_2_true-unreach-call.c [OCRS][without] 2 PASS
0.331
rec-count_by_k_true-unreach-call.c [OCRS][without] 2 PASS
0.363
rec-count_by_nondet_true-unreach-call.c [OCRS][without] 2 FAIL
0.389
rec-gauss_sum_true-unreach-call.c [OCRS][without] 2 PASS
0.368
rec-half_true-unreach-call.c [OCRS][without] EXCEPTION
0.337 (0.401)
rec-nested_true-unreach-call.c [OCRS][without] 3 PASS
0.613
Total Below Time = 3.073 (was 3.104)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/recurrences
cubic_polynomial.c [OCRS][without] 2 PASS
0.533 (0.484)
edit_distance_bottom_up.c [OCRS][without] 3 TIMEOUT
300.007
edit_distance_top_down.c [OCRS][without] 3 FAIL
50.571
log_log_n_solution.c [OCRS][without] 3 FAIL
0.422 (0.636)
log_n_solution.c [OCRS][without] 2 FAIL
0.451
multivariate_1.c [OCRS][without] TIMEOUT
300.007
multivariate_higher_order.c [OCRS][without] 7 FAIL
FAIL
FAIL
25.885
n_cubed_solution.c [OCRS][without] EXCEPTION
0.329
n_log_n_solution.c [OCRS][without] TIMEOUT
300.034 (1.882)
n_squared_two_base_case_even.c [OCRS][without] 2 PASS
0.375
n_squared_two_base_case_odd.c [OCRS][without] 2 PASS
0.4
pascals_dynamic.c [OCRS][without] 3 FAIL
20.228
pascals_iterative.c [OCRS][without] 1 FAIL
5.374 (6.26)
pascals_recursive.c [OCRS][without] TIMEOUT
300.005
squared_solution.c [OCRS][without] TIMEOUT
300.16 (12.112)
two_n_even.c [OCRS][without] 2 PASS
0.395
two_n_odd.c [OCRS][without] 2 PASS
0.38
two_to_n_over_two_even.c [OCRS][without] TIMEOUT
300.034 (44.691)
two_to_n_over_two_odd.c [OCRS][without] TIMEOUT
300.026 (44.264)
two_to_n_squared.c [OCRS][without] 8 TIMEOUT
300.007 (133.338)
two_to_two_to_n.c [OCRS][without] 7 FAIL
1.799 (1.539)
Total Below Time = 2507.422 (was 1242.925)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/exponential
adding_and_mult.c [OCRS][without] 1 PASS
0.343
exp_add_loop_rec.c [OCRS][without] 1 FAIL
0.396 (0.348)
exp_add_loop_variable.c [OCRS][without] 1 PASS
0.385
exp_with_linear_inner_loop.c [OCRS][without] 1 FAIL
0.422
linear_two_to_n.c [OCRS][without] 8 FAIL
1.592 (1.339)
loop_five_to_n.c [OCRS][without] 1 FAIL
0.376 (0.337)
non_loop_five_to_n.c [OCRS][without] 8 FAIL
10.386 (8.639)
power_log.c [OCRS][without] 1 PASS
0.416
power_log_modified.c [OCRS][without] 1 PASS
0.473 (0.393)
simple_exponential.c [OCRS][without] 1 PASS
0.354
simple_exponential_for.c [OCRS][without] 1 PASS
0.353
two_to_n_minus_n.c [OCRS][without] 8 FAIL
1.825
two_to_n_minus_n_loop.c [OCRS][without] 8 FAIL
2.884
Total Below Time = 20.205 (was 17.495)

>25% better
25%-10% better
No significant change
10%-25% worse
>25% worse