Mantis Bugtracker

Einfache Problemansicht anzeigen Zu Notizen wechseln ] erweiterte Anzeige ] Problem-Historie ] Drucken ]
ID Kategorie Auswirkung Reproduzierbar Meldungsdatum Letzte Aktualisierung
0000059 [Bisect] general Unschönheit immer 2010-07-06 15:54 2011-02-12 14:45
Reporter art1 Anzeigestatus öffentlich  
Bearbeitung durch xclerc
Priorität normal Lösung erledigt  
Status erledigt   Produktversion 1.0
Zusammenfassung 0000059: instrumented code is 20times slower
Beschreibung Hello,

The instrumented testcases runs 20 times slower then without bisect instrumentation. I tried this to test the simple median-function:

(** median of a list
  * @param lst list of ints
  * @return median
  *)
let int_median (lst: int list) =
  match lst with
      [] -> 0
    | _ ->
        begin
          let ar = Array.of_list lst in
          let len = (Array.length ar) in
          let pivot = (len-1)/2 in
            Array.sort Helper.int_cmp ar;
            if len mod 2 == 0 then
              round_int (float_of_int (ar.(pivot) + ar.(pivot+1)) /. 2.0)
            else
              ar.(pivot)
        end
;;

My testcase was:
let testplan_int_median =
  "math (int_median) " >::: [
    "empty " >:: (fun () -> !==! (int_median []) 0);
    "simple " >:: (fun () -> !==! (int_median [1]) 1);
    "2 positives " >:: (fun () -> !==! (int_median (1::3::[])) 2);
    "2 same " >:: (fun () -> !==! (int_median (1::1::[])) 1);
    "2 negatives " >:: (fun () -> !==! (int_median (~-1::-3::[])) ~-2);
    "2 values " >:: (fun () -> !==! (int_median (~-1::1::[])) 0);
    "3 positives " >:: (fun () -> !==! (int_median (1::3::2::[])) 2);
    "3 negatives " >:: (fun () -> !==! (int_median (~-1::~-3::~-2::[])) ~-2);
    "3 values " >:: (fun () -> !==! (int_median (~-1::1::0::[])) 0);
    "5.000.000 values " >:: (fun () -> !==! (int_median five_mill_int) 1);
  ]
;;

I am using OUnit and the testplan runs 15s on my Athlon64 without using bisect. With bisect switched on, the testplan needs approximately 310s.

Because I have larger testcases that issue makes it hard to run all my tests nightly and to calculate the test coverage.

If you have some useful hints, it would be fine.

Thanks in advance, Bye Andreas

Zusätzliche Information
Tags Keine Tags zugeordnet.
Angehängte Dateien

- Problem-Beziehungen

-  Notiz
(0000210)
xclerc (Administrator)
2010-09-26 14:52

Well, I am afraid that a huge slowdown is unavoidable when
using code coverage, as one has roughly to "increment counters
everywhere". As far as I can tell, clever profiling approaches
(read "sampling") cannot be applied to code coverage.

Here a some additional notes:
  - it would be possible for Bisect to give additional control over the way
     source code is instrumented (some kind of filter to instrument only some
     given code parts, or some given code constructs - the currently only
     available granularity being the file);
  - it is possible to use the aforementioned file-based granularity to use
     code coverage for unit testing rather than for integration testing (meaning
     that at each run only the tested module is instrumented, the remaining
     modules being "uninstrumented");
  - only the tested code need to be instrumented, the testing code (i. e. test
     framework, and tests encoding) should not be instrumented - this can be
     particularly costly if your testing code contains loops and/or is random-based.

However, I cannot tell if these suggestions can be applied to your settings.


I would be glad to read your feedback about the above.

- Problem-Historie
Änderungsdatum Benutzername Feld Änderung
2010-07-06 15:54 art1 Neues Problem
2010-09-26 14:52 xclerc Problemnotiz hinzugefügt: 0000210
2010-09-26 14:52 xclerc Bearbeitung durch => xclerc
2010-09-26 14:52 xclerc Status neu => Rückmeldung
2011-02-12 14:45 xclerc Status Rückmeldung => erledigt
2011-02-12 14:45 xclerc Behoben in Version => 1.1
2011-02-12 14:45 xclerc Lösung offen => erledigt


Mantis 1.1.7[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker