EzDevInfo.com

Generating sorted data with test.check

I'd like to use test.check to generate sorted time series data of the form

[ [timestamp value] [timestamp value] ..]

where the timestamp, value -pairs are in ascending order by the timestamp.

I can easily generate such data in random order with

(gen/tuple timestamp gen/int) where timestamp is e.g. (gen/choose 1412664660 1423419720)

How should I go about generating sorted data?

Generating structured maps with test.check

I'm playing around with test.check, and I'm testing a function which takes a map as an argument. These maps do have a defined structure, such as:

{:name "Bob" :age 42 :email "bob@example.com" :admin true}

Key point, there is a set of expected keys, the values of which have differing clearly defined generators.

I took a look at gen/map, but it's not obvious how to use it for more structured key/value pairs:

(gen/sample (gen/map gen/keyword gen/boolean) 5)
;; => ({} {:z false} {:k true} {:v8Z false} {:9E false, :3uww false, :2s true})

This seems like a simple scenario, but I can't find an example.

How can I generate structured maps, such as the one described here, using test.check?

How do I create a test.check generator for a sequence of actions in a constrained order?

(require '[clojure.test.check.generators :as gen])

(def ACTIONS
  {:create-new-user #{}
   :edit-user #{:create-new-user}
   :create-new-board #{:create-new-user}
   :edit-board #{:create-new-board}
   :create-new-anonymous-comment #{:create-new-board}
   :create-new-signed-comment #{:create-new-board}
   :edit-comment-text #{:create-new-anonymous-comment :create-new-signed-comment}
   :edit-comment-text-and-flip-anonymity #{:create-new-anonymous-comment :create-new-signed-comment}
   :flip-anonymity #{:create-new-anonymous-comment :create-new-signed-comment}
   :vote-comment-up #{:create-new-anonymous-comment :create-new-signed-comment}
   :vote-comment-down #{:create-new-anonymous-comment :create-new-signed-comment}})

(def actions (-> ACTIONS keys vec gen/elements gen/vector))

(defn filter-actions-into-logical-order [as]
  (let [seen (atom #{})]
    (filter
     (fn [v]
       (let [required (get ACTIONS v)
             valid? (or (some? (some required @seen)) 
                        (and (empty? @seen) (= v :create-new-user)))]
         (when valid?
           (swap! seen conj v)
           true)))
     as)))

(def ordered-actions (gen/fmap #(-> % filter-actions-into-logical-order vec)  actions))

As an example of the two generators:

# (last (gen/sample actions 100))
[:edit-user :vote-comment-down :flip-anonymity :vote-comment-down :vote-comment-down :vote-comment-up :edit-user :create-new-anonymous-comment :edit-board :create-new-signed-comment :vote-comment-up :edit-comment-text-and-flip-anonymity :edit-user :create-new-signed-comment :edit-user :edit-user :vote-comment-down :edit-user :vote-comment-down :create-new-user :vote-comment-down :create-new-user :create-new-user :edit-comment-text-and-flip-anonymity :create-new-user :edit-comment-text-and-flip-anonymity :create-new-anonymous-comment :edit-comment-text :create-new-board :vote-comment-down :flip-anonymity :create-new-signed-comment :vote-comment-up :create-new-user :create-new-signed-comment :edit-user :create-new-user :create-new-board :vote-comment-down :create-new-board :create-new-board :create-new-board :edit-board :edit-comment-text-and-flip-anonymity :edit-user :edit-comment-text :create-new-signed-comment :vote-comment-up :edit-comment-text-and-flip-anonymity :flip-anonymity :create-new-anonymous-comment :create-new-anonymous-comment :edit-board :create-new-signed-comment :edit-comment-text-and-flip-anonymity :edit-board :vote-comment-up :edit-comment-text :create-new-board :edit-comment-text-and-flip-anonymity :create-new-board :vote-comment-down :edit-comment-text-and-flip-anonymity :vote-comment-up :create-new-user :vote-comment-up :edit-comment-text :edit-board :edit-comment-text-and-flip-anonymity :flip-anonymity :edit-board :create-new-anonymous-comment :flip-anonymity :create-new-signed-comment :edit-user :edit-comment-text-and-flip-anonymity :edit-comment-text :edit-comment-text :create-new-user :flip-anonymity :edit-user :vote-comment-up :edit-user :create-new-user :edit-comment-text :edit-comment-text :flip-anonymity :edit-comment-text :edit-board :flip-anonymity :edit-board :edit-comment-text :edit-user :create-new-user :flip-anonymity]


# (last (gen/sample ordered-actions 100))
[:create-new-user :edit-user :edit-user :create-new-board :edit-board :edit-user :create-new-anonymous-comment :edit-comment-text :edit-board :edit-user :edit-user :vote-comment-up :edit-comment-text :create-new-signed-comment :edit-comment-text :create-new-board :edit-comment-text :edit-comment-text :edit-comment-text :vote-comment-up :vote-comment-up :edit-board :edit-comment-text-and-flip-anonymity :create-new-signed-comment :create-new-anonymous-comment :create-new-signed-comment :edit-user :create-new-anonymous-comment :edit-board :create-new-board :create-new-anonymous-comment :create-new-board :flip-anonymity :create-new-anonymous-comment :edit-board :vote-comment-up :vote-comment-down :edit-board :edit-comment-text :edit-user :edit-comment-text :flip-anonymity :create-new-signed-comment :vote-comment-up :edit-comment-text-and-flip-anonymity :vote-comment-up :create-new-signed-comment :edit-comment-text :create-new-signed-comment :create-new-anonymous-comment :edit-board :create-new-anonymous-comment]

ACTIONS is a map where the key is the name of the action, and the value is the (OR based) dependency for that action. As an example, you must first :create-new-user before you can do anything, you must :create-new-board before you can :edit-board, and you must have at least one :create-new-*-comment before you can :edit-comment-text.

The code above seems to work, but it is ugly. 1) I don't like how filter-actions-into-logical-order code has to have a specific exception for :create-new-user. 2) I don't like that I am basically taking the list of random actions, and filtering it down until the actions make ordered sense.

I am wondering how others would generate a sequence of actions like this using test.check? Surely there must be a way to do so using only generators?

How do you generate collections with a specific property (like stddev) using test.check

I want to use clojure's test.check library to generate collections that I can do some simple stats on, things like computing mean, stddev, confidence intervals and that sort of thing.

How can I generate the collections so that they have pre-determined values for these properties?

How to generate UUIDs that can work with test.check in Clojure

Generative testing seems interesting, but I needed to generate random UUIDs as part of the testing. java.util.UUID/newRandom doesn't play nice with test.check shrinking.

The java code looks like:

public static UUID randomUUID()
{
  long lsb = r.nextLong();
  long msb = r.nextLong();

  lsb &= 0x3FFFFFFFFFFFFFFFL;
  lsb |= 0x8000000000000000L; // set top two bits to variant 2

  msb &= 0xFFFFFFFFFFFF0FFFL;
  msb |= 0x4000; // Version 4;

  return new UUID( msb, lsb );
}

Which is trickier to translate to Clojure than it would seem.

How do I write a random UUID function in Clojure that can be successfully shrunk?

clojure.test.check poor performance of nested gen/list generators

I'm trying to test a Record data structure that contains nested lists. ie, a Document contains many Sentences, and each Sentence contains many Words. I created a Word generator, Sentence generator and Document generator, but the performance is very slow. Is there any way I can improve the performance?

Here's a simplified example:

(def p (prop/for-all [coll (gen/list (gen/list (gen/list gen/nat)))] (= 1 1)))
(tc/quick-check 100 p)

Takes nearly 10 seconds.

That can be improved by limiting the size of each internal list using vector:

(def p (prop/for-all [coll (gen/vector (gen/vector (gen/vector gen/nat 0 25) 0 25) 0 25)] (= 1 1)))

That only takes 1.5 seconds (still quite slow), but it violates the semantics that I'm trying to test. I could live with that, but when I try to start generating record constructors, the performance problems return worse than before, even with the restricted vector sizes.

Here's a realistic example.

(defmacro defgen [name ctor & body]
  `(def ~name
     (gen/fmap (partial apply ~ctor)
               (gen/tuple ~@body))))

defgen takes a binding symbol, a Record constructor, and applies the constructor to the body of the macro. This is copied from the examples here:

https://github.com/clojure/test.check/blob/master/doc/intro.md#record-generators

Then it's used as follows:

(defgen word-gen 
  ->Word 
  gen/string-alphanumeric)

(defgen sentence-gen
  ->Sentence
  (gen/vector word-gen 0 25))

(defgen doc-gen
  ->Doc
  gen/string-alphanumeric
  (gen/vector sentence-gen 0 25))

(def p (prop/for-all [docs (gen/vector doc-gen 0 25] (= 1 1)))
(quick-bench (tc/quick-check 100 p))

Mean-execution time: ~several minutes..

Now, if I reduce each individual generator down to a max of 10, and I reduce the quick-check size from 100 to 50, I can get it to execute in about a second or two. But, combined with all the other tests, this is still way too slow. What's worse, I've limited the test cases so much compared to the real world so as to no longer be that "generative". (ie, it's common to have hundreds of sentences in a document).

Thanks.