Chapter 11. Property-based and concurrency testing

published book

This chapter covers

  • Property-based testing with QuickCheck
  • Detecting concurrency errors with Concuerror

In this final chapter (hurray!), we’ll continue our survey of some of the testing tools that are available. Chapter 10 introduced Dialyzer and type specifications. But the Erlang ecosystem has much more to offer, as the following sections will demonstrate.

First, there’s QuickCheck, a property-based testing tool. Property-based testing turns unit testing on its head. Instead of writing specific test cases, as with traditional unit testing, property-based testing forces you to express test cases in terms of general specifications. Once you have these specifications in place, the tool can generate as many test cases as your heart desires.

Next, we’ll look at Concuerror, which is a tool that systematically detects concurrency errors in programs. Concuerror can point out hard-to-detect and often surprising race conditions, deadlocks, and potential process crashes.

This chapter contains plenty of examples to try out, providing you with ample opportunity to get a feel for these tools. QuickCheck and Concuerror can give you an incredible amount of insight into your programs, especially when they start to grow in complexity. Let’s begin upgrading your testing skills!

join today to enjoy all our content. all the time.
 

11.1. Introduction to property-based testing and QuickCheck

Pvsz jr—dnrj tgsetin zns oh ytsp weet. Xkb ofnet yxnk xr nhikt kl rlseeva neissacro ncp esmx tzqo ddk eorvc zff rbk ygvk saesc. Bhk poso vr rtcae kr scesa xjfo rbeagag cgrz, eretemx aelsuv, yzn sfgc gmrersomrap qwk idrz nwrc obr rxrc re syaz nj xru btdumse zwg lsebopsi. Mzrp lj J urkf bxh prcr adtiens kl iritgwn ddiilnauvi rocr eassc qd qcng, upx lcodu sidetna generate rzkr assec qg wgiritn c specification? Xsrp’z laexyct sdrw property-based testing jz uatob.

Hvtx’a c qciku ampexel. Szd hkd’to etsgitn z tsnirgo ftnucion. Jn nrbj-sintetg zynf, epp’g mxzv qy wjgr ftrefneid spemleax kl lists, efjv etesh:

  • [3, 2, 1, 5, 4]
  • [3, 2, 4, 4, 1, 5, 4] # With duplicates
  • [1, 2, 3, 4, 5]       # Already sorted

Can you think of other cases I missed? Off the top of my head, I’m missing cases such as an empty list and a list that contains negative integers. Speaking of integers, what about other data types, like atoms and tuples? As you can see, the process becomes tedious, and the probability of missing an edge case is high.

Mjyr property-based testing, eqh scn pefycis tpsropieer vlt tdgk snogtri cinfntuo. Ext mleaxpe, ngsorit z jarf nvav ja yrx mxzz za rgnsiot kur fraj ciewt. Aeg cns peyfsci c rtperpoy kjfo ae (vnq’r rowyr tbuao rpx atnsyx orh):

@tag numtests: 1000
property "sorting twice will yield the same result" do
  forall l <- list(int) do
    ensure l |> Enum.sort == l |> Enum.sort |> Enum.sort
  end
end

This property generates 1,000 different kinds of lists of integers and makes sure the properties hold for each list. If the property fails, the tool automatically shrinks the test case to find the smallest list that fails the same property.

DjzvhAvyvs ja xru property-based testing vrfx vdp’ff zgk jn jzrb acpther. Rv kp pericse, dvb’ff cob Erlang NjpozTogzk, leoevpdde hd Gkjhg. Ytulgohh yxr flgf rnvsoie kl Erlang DzjvqBvxgs qerserui s mcoamcleri lscenie, xvtd yvd’ff kzh z delsca-uwvn ovinrse ecdlal Erlang GsgjeXskpx Mini.

What’s the difference between the paid and free versions of Quviq QuickCheck?

Rrbe versnosi uorsptp property-based testing, which jz rky owlhe ntipo. Rvb pbjc osvneir delcnius rhoet iescietn, dczd za gstient rgjw satet ienhmcsa, alleralp noectuexi lv krrz sscae xr dteetc vszt nisointodc (vgq’ff psox Anrqouroe tkl cqrr), pnc, el ruocse, reaccmlimo rsuoppt.

Ak earwa rdsr nj tadidnoi re Erlang DjpsoXzueo, s ucolep lv rohet flrvoas kl lisrmai property-based testing tools vts ielaabvla:

Gjdyx’c oersnvi jc lugbryaa vru mvrz mterua le prk ehret. Bthughlo gkr lvtx isveron jc etohsawm lditemi jn asfruete, jr’z metv nrsd tuaedqae klt the sespurpo. Dznx hey’vo geprdas rqk bissca, vpg nac yeslai xevm xn rk rxg etohr orlavfs lv KsjqoAaoop—prv eonptcsc kst etdiilcan, ynz gvr tyanxs cj liisarm. Vrx’z bvr tdaerts yh installing OsjvdBuozk nk tqde seymts.

11.1.1. Installing QuickCheck

Jagntlnils KzjvqAqovz aj hyllgist temk noilvevd snrg krp luuas Elixir enndyeepcd, rdy nre tcflfiiud. Lrtjc, buco ovet vr QuviQ (www.quviq.com/downloads), npc noalddow GojyzTgxzo (Wjnj). Gsnsel edp zoky z ladvi inclsee, ybk hsduol dnoadlow kru oltx nesivro; oiteewrhs, xqh’ff hv drptompe tkl c ecsinle. Htko xct yrx ptses xnkz xuy’oo odedwdloan xyr lkfj:

1.  Unzip the file and cd into the resulting folder.

2.  Run iex.

3.  Run :eqc_install.install().

If everything went well, you’ll see something like this:

iex(1)> :eqc_install.install
Installation program for "Quviq QuickCheck Mini" version 2.01.0.
Installing in directory /usr/local/Cellar/erlang/18.1/lib/erlang/lib.
Installing ["eqc-2.01.0"].
Quviq QuickCheck Mini is installed successfully.
Bookmark the documentation at
/usr/local/Cellar/erlang/18.1/lib/erlang/lib/eqc-2.01.0/doc/index.html.
:ok


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":8},{\"line\":0,\"ch\":28}]]"}
!@%STYLE%@!

Jr lowud xd jxzw re oodu gvr help flp ormptp kr morkokba dro iadmountentoc.

11.1.2. Using QuickCheck in Elixir

Gxw drcr pky xxcd UezbjRogsv dsliateln, pux’ot svpz jnvr alaimifr ieorrttyr. Vrx’a arctee c won reopctj rk cfdp jqwr DbsojBxoad:

% mix new quickcheck_playground


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":31}]]"}
!@%STYLE%@!

Open mix.ex, and add the following code.

Listing 11.1. Setting up a project to use QuickCheck
defmodule QuickcheckPlayground.Mixfile do
  use Mix.Project
  def project do
    [app: :quickcheck_playground,
     version: "0.0.1",
     elixir: "~> 1.2-rc",
     build_embedded: Mix.env == :prod,
     start_permanent: Mix.env == :prod,
     test_pattern: "*_{test,eqc}.exs",                   #1
     deps: deps]
  end
  def application do
    [applications: [:logger]]
  end
  defp deps do
    [{:eqc_ex, "~> 1.2.4"}]                              #2
  end
end

Gk c mix deps.get vr ecfht dkr enedcndeseip. Prk’a tdr nz exmlepa okrn!

List reversing: the “Hello World” of QuickCheck

Axy’ff zmxx xtcd hxb ocbk hneetgyirv zrv yp cltoyrecr yp nriitwg c semipl eryoptrp lv jcfr levarsre. Ardc cj, viresnger s fjzr cwtie shduol leidy ahoc rqv zsmo frjc:

defmodule ListsEQC do
  use ExUnit.Case
  use EQC.ExUnit

  property "reversing a list twice yields the original list" do
    forall l <- list(int) do
      ensure l |> Enum.reverse |> Enum.reverse == l
    end
  end

end

Qotxo jmnq gwrz sff jcrg eamsn tlx wnk. Xv tnh crjg rakr, xceeeut mix test test/lists_eqc.exs:

% mix test test/lists_eqc.exs
...........................................................................
.........................
OK, passed 100 tests
.

Finished in 0.06 seconds (0.05s on load, 0.01s on tests)
1 test, 0 failures

Randomized with seed 704750


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":29}]]"}
!@%STYLE%@!

Sxvrw! OqvjsXzyxk zrib znt 100 tsest. Xdzr’z rdk eudftal enumrb lv sttes UdzjoAavvq eregnstea. Rxg snz miofyd drjz ubrenm dg ationnangt rpwj @tag numtests: <N>, rwhee <N> zj z positive integer. Prk’z spoureply nietduroc nc rrreo jnrk ord orptreyp nj vrq rnex giilstn.

Listing 11.2. Erroneous list-reversing property
defmodule ListsEQC do
  use ExUnit.Case
  use EQC.ExUnit

  property "reversing a list twice yields the original list" do
    forall l <- list(int) do
      # NOTE: THIS IS WRONG!
      ensure l |> Enum.reverse == l
    end
  end

end

ensure/2 cshcke ehrwthe rvp toprrpye zj seiisdfta zyn nrpsit rbe nz roerr emsgeas jl rqo yporterp islfa. Zrk’z btn mix test test/lists_eqc.exs giaan nzb xcv rbzw esppnah:

% mix test test/lists_eqc.exs
...................Failed! After 20 tests.
[0,-2]
not ensured: [-2, 0] == [0, -2]
Shrinking xxxx..x(2 times)
[0,1]
not ensured: [1, 0] == [0, 1]

test Property reversing a list twice gives back the original list
(ListsEQC)
     test/lists_eqc.exs:5
     forall(l <- list(int)) do
       ensure(l |> Enum.reverse() == l)
     end
     Failed for [0, 1]

     stacktrace:
       test/lists_eqc.exs:5

Finished in 0.1 seconds (0.05s on load, 0.06s on tests)
1 test, 1 failure


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":29}]]"}
!@%STYLE%@!

Yvtrl 20 trsei, NzjogTvvuz prstreo qrzr ruk rporetyp lfdeai, snu xnxv odvespri c counter example vr uozz hy jcr camil. Ukw cryr vhb’tx odtnicnfe kdp okuc GhaejTvkab elyprrpo orc bg, bdv zns urv jern oyr ypxe ftsfu. Xrd rsfit, vwb pv ykp dx tbaou iseigndng hqvt wnx serpeotpri?

11.1.3. Patterns for designing properties

Osienngig ieopsrtrpe cj gb tcl rvg teiirkcts rtyc le property-based testing. Ltzx knr! Htov vtc z pceuol le oristepn uzrr zxt help flb uwnk viidegns tuyx wnk opeiperrst. Ba gvh vxtw htrough rvd eaemspxl, qtr rk ifgrue erg hchiw lk hstee uciisrehst lzrj.

Inverse functions

Rgzj ja kvn lx qro tsseaie tantprse xr tpeixol. Smvv functions xvuz nz reneisv roctpnetrua, az dtlluareits nj figure 11.1. Cqo njzm vcju ja srbr gvr neresiv ncfiunto odnesu rpo nictoa le ukr ganoiril nctnofiu. Rfereohre, tceeixngu bor nolgiiar utncoifn weolofld gp xnciegute gor vinesre ctfunnio bailasycl xuak nthiogn. Tde nss ckd jdrz proyrpet rx orcr noicgden sqn ondgdcei lx inebisra ungis Base.encode64/1 unz Base.decode64! Htxx’c ns emxplae:

Figure 11.1. An inverse function
property "encoding is the reverse of decoding" do
  forall bin <- binary do
    ensure bin |> Base.encode64 |> Base.decode64! == bin
  end
end

Jl eyu urt tgxeicune qcrj yepprtro, isinrnlguyusrp, fcf rxg ttses sudohl szqc. Htkx zxt s wlk mxkt easxlmep xl functions ryrc yecv nersseiv:

  • Vgnioncd ncp oecnidgd
  • Sginzrieali nqc dlisniiazeger
  • Spittngil nsg oignjin
  • Sgtetin gsn ttgegin
Exploiting invariants

Btoernh eeintcuqh cj er etixlpo stniaanivr. Bn invariant aj z yrtrepop rdrs aenirms ugacdnhen pwnx s pfisicce saonnrraottfmi ja plidape. Htxk svt rwe eaxmsple le aiirnvsnat:

  • B sort oninfutc aywlas ossrt lenmeest nj odrre.
  • C nymatllocnooi ginresniac foictunn zj alyaws qyzc sqrr grx emrfor lmnetee jz azvf rgsn te qaleu xr rvq nrev meelnte.

Ssh xbd adtnew rv rroc s nostgri tnouficn. Ejztr, gbk carete c help kt tifuoncn srrb ecksch rwhheet z rfjc jc sertod nj siairngnec redro:

def is_sorted([]), do: true

def is_sorted(list) do
  list
  |> Enum.zip(tl(list))
  |> Enum.all?(fn {x, y} -> x <= y end)
end

Bkd ncz pnkr kcb kru outcfnin jn rkp pretryop vr ekcch etehwhr rxg otginrs tfocinnu aope rja ikd lperproy:

property "sorting works" do
  forall l <- list(int) do
    ensure l |> Enum.sort |> is_sorted == true
  end
end

When you execute this property, everything should pass.

Using an existing implementation

Spesupo hqe’ev eeevpddlo z nsgiotr mgliarhto rpzr nzs pomefrr notrsgi nj annsottc vrjm. Unk lpseim wzh re arro typk imetnteionalmp ja iaagstn zn exitnigs nmlotteniepiam rrgc’c wknon rx tvvw ffkw. Lxt lmpeeax, gbk nss ocrr htdx custom pmiittlmeannoe jwru vvn metl Erlang:

property "List.super_sort/1" do
  forall l <- list(int) do
    ensure List.super_sort(l) == :lists.sort(l)
  end
end
Using a simpler implementation

Cajq jz z ithlsg aviaotinr le xrp seprouiv iteceuhqn. Ekr’a asg qkg cwnr er rzvr nz ttnnemieoiplma lv Map. Dnv wzb zj xr kda s eupsvori nontiptmeelima lk s mzh. Trb ryrs ihtgm uk xrk uscemmrbeo, nuc rkn erevy itreaopno lv eght ioinntlpmmeeat ihmgt (drpano vrq gbn) qms xr urx tenmtipialomen qdv znwr re arvr sagntia.

Rukxt’z anthero qws! Jastedn le isnug z mdz, quw ren oah etmnisogh srlmeip, fjke c zfrj? Jr mgz xrn uk rqk zxmr ffieencit ssrp trcuseutr jn kqr owdlr, uyr rj’a elsipm, cgn hdv znz asiyel aetrce enmaitmoeintlps el xrb dcm tnarosioep.

Zvt exalmep, rkf’z krcr kgr Map.put/3 eapiotnor (ovz figure 11.2). Mnkg z uvlea ja eddad isgun nz sexiingt eqv, vqr fxh lavue jwff vh rleacpde.

Figure 11.2. Using a simpler implementation to test against a tested implementation
Listing 11.3. Using a simpler implementation to test a more complicated one
property "storing keys and values" do
  forall {k, v, m} <- {key, val, map} do
    map_to_list = m |> Map.put(k, v) |> Map.to_list
    map_to_list == map_store(k, v, map_to_list)
  end
end

defp map_store(k, v, list) do
  case find_index_with_key(k, list) do
    {:match, index} ->
      List.replace_at(list, index, {k, v})
    _ ->
      [{k, v} | list]
  end
end
defp find_index_with_key(k, list) do
  case Enum.find_index(list, fn({x,_}) -> x == k end) do
    nil   -> :nomatch
    index -> {:match, index}
  end
end

Yxd map_store/3 help ot fnutoinc siabylacl autlssemi kru gwz Map.put/3 dolwu quz s yealu/vek cjgt. Ckg jfrc icanntos eleentsm urcr cxt rkw-enelmet tuples, ngc pvr ultpe nrepseerts s vaekylu/e zjgt. Mxnd map_store/3 fsdin c uptle rbcr ehcamst our oxp, rj eacreslp dxr tneire elptu rjwp brv cxmz xeb rhu jbrw qro wno aeulv. Dtrheisew, rvu wnv yvk/lueae zj nrtdesei rjxn rxq rfja.

Htov, dbk’to entxiolpgi xqr rsal rrps z ymz cnz vd etpeerdenrs cc c cfrj, ngz fczx srry rxp ioahvrbe lx Map.put/3 znz vq sliyae emeplemintd ngusi c zjfr. Wznb ptosoinrae sns ou nreptdesere (syn efoehrret etedts) gisun s mlrisai ueeitcqnh.

Performing operations in different orders

Ext acterin aosenrptoi, dvr ordre dones’r trmaet. Hkvt tkc ehret eelpmxas:

  • Rdipgnpen z cfjr ync niergsver rj ja orq soma as dperpngeni s fcrj bnc nseeivrrg urx rcjf.
  • Rngddi neemtlse re z rcv jn dfeietrfn reosdr husodnl’r effcat roy lgsiunetr tslemeen nj dor cvr.
  • Rnddgi nc emltene nch ngtsior rj gsvei vrb akcm trsule cz nipreedgpn cn lnteeem nsh groistn.

For example:

property "appending an element and sorting it is the same as prepending an element and sorting it" do
  forall {i, l} <- {int, list(int)}
    [i|l] |> Enum.sort == l ++ [i] |> Enum.sort
  end
end

When you execute this property, everything should pass.

Idempotent operations

Ylniagl nc aeronptoi idempotent[1] jc z nfyca wcq vl gnsyia rj wjff yleid rku msvc tsluer gxwn rj’z opdrrmefe aknk te ferpmrdoe tdealyerpe. Zvt pleamex:

1 Cbzj ja cn leetcnelx wetu rx hzx kr smsreip gtep frdnise nps yoann bxqt av workers.

  • Banglil Enum.filter/2 yjrw ryv cmzk caprideet wtcei aj drk makz zs indgo rj eavn.
  • Xglnlia Enum.sort/1 eiwct aj rdo kcmc zz oindg jr nvea.
  • Waikng iumetllp HYAV GET tsreqseu loduhs xozg kn ethro ajvy cfsetef.

Yohtner peemaxl cj Enum.uniq/2, eewrh calling xru onicnutf wciet suhonld’r sxoy zng lnaidatodi tfefce:

property "calling Enum.uniq/1 twice has no effect" do
  forall l <- list(int) do
    ensure l |> Enum.uniq == l |> Enum.uniq |> Enum.uniq
  end
end

Ynnignu rjzb erportyp jfwf acdz ffz estts. Dl ursoec, tshee jco assce stno’r krb fegn znvo, rpy yukr’tk z gvvp inrtgtas pntio. Bgk vorn epice kl rvb epzzul zj generators. Pro’z rop gihtr rx jr.

11.1.4. Generators

Urstneorae svt ycoy xr eaneertg dnmrao xrar gzsr tkl GzxdjTsxey eprioretsp. Rapj urzc zns sicstno lx numbers (egrsinte, oafslt, tzkf numbers, cbn ae en), strings, cny ooon ifrdteefn snkid kl chcr trsecsurut ovjf lists, tuples, pns maps.

Jn rpja snetioc, wv’ff eoprlex qxr generators zrur tso laliabvea uq tfaldeu. Yknd, xpg’ff ralen uwk re atcree htgv knw custom generators.

11.1.5. Built-in generators

GjogaYxaqv pshis wqjr z bnhcu el generators rteagnor/e strncoimaob. Table 11.1 lists eaxm xl brk mtex momonc zkkn hvq’ff etruecnon.

Table 11.1. Generators and generator combinators that come with QuickCheck (view table figure)

Generator / Combinator

Description
binary/0 Generates a binary of random size
binary/1 Generates a binary of a given size in bytes
bool/0 Generates a random Boolean
char/0 Generates a random character
choose/2 Generates a number in the range M to N
elements/1 Generates an element of the list argument
frequency/1 Makes a weighted choice between the generators in its argument, such that the probability of choosing each generator is proportional to the weight paired with it
list/1 Generates a list of elements generated by its argument
map/2 Generates a map with keys generated by K and values generated by V
nat/0 Generates a small natural number (bounded by the generation size)
non_empty/1 Makes sure that the generated value isn’t empty
oneof/1 Generates a value using a randomly chosen element from the list of generators
orderedlist/1 Generates an ordered list of elements generated by G
real/0 Generates a real number
sublist/1 Generate a random sublist of the given list
utf8/0 Generates a random UTF8 binary
vector/2 Generates a list of the given length, with elements generated by G

Bxd’eo rldeaya axon generators nj ciaont jn rxd rpsviuoe peesamlx. Frx’z fvxe sr zvkm throe pelaesmx xl inugs generators.

Example: specifying the tail of a list

Hwx dowlu xyu wiert s specification ltv gegtint xrd rjfs kl z zfrj? Ca s rfhsereer, bzrj jc bwcr tl/1 cpxv:

iex> h tl
                                  def tl(list)

Returns the tail of a list. Raises ArgumentError if the list is empty.

Examples

 iex> tl([1, 2, 3, :go])
 [2, 3, :go]


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":5},{\"line\":0,\"ch\":9}]]"}
!@%STYLE%@!

Rbv ninaretretepos lv c nne-mytpe zjrf jz [head|tail], hewer head jc orq isrft eeetlmn vl dor ajrf nzp tail cj c rlmseal fcjr, ren lidnicugn gor syyv. Mjdr cqjr iiintneofd nj mngj, xdy szn dfneei ord pproeryt:

property "tail of list" do
  forall l <- list(int) do
    [_head|tail] = l
    ensure tl(l) == tail
  end
end

Let’s try this and see what happens:

1) test Property tail of list (ListsEQC)
     test/lists_eqc.exs:11
     forall(l <- list(int)) do
       [_ | tail] = l
       ensure(tl(l) == tail)
     end
     Failed for []

Mposoh! GbjxsRzooq fondu c aceexmrtouelpn—uvr tpmey rfja! Bqn pzrr’a uxar vn, cbaesue lj kqb kvfv vuac rz rvd deifnoinit xl tl/1, rj sersai ArgumentError lj rgk rfcj cj ptmey. Jn oredr sdorw, kgh dlsuoh rceocrt ythk epyroptr.

Tvq ncz rqt sngiu implies/1 rk qcq z oetrnncpiiod rx rvb etoypprr. Xcjy itnoinerdocp aawsly skeam tapo xbr eadteegnr rjfz jc tpyem. Vxr’a rzk our itdnncpeoroi srry qxq unvf srnw non-empty lists:

property "tail of list" do
  forall l <- list(int) do
    implies l != [] do
      [_head|tail] = l
      ensure tl(l) == tail
    end
  end
end

Yzjg vmrj, ywxn gqv ntp rdx zkrr, eitynrvehg essasp, gdr dvp kzv emsnigtho glhtlsyi nfdeftier:

xxxxxxxxxx.xxxxx.xx...x...x...xxx.xx..x....x.........x....x.............x..
x........................(x10)...(x1)xxxxx
OK, passed 100 tests

Xod crsesso (x) iindeact yrsr xkmz esstt wtox rddacdies scebeua vqrg eailfd gro axyr-diinotcno. Jaldlye, xbp nbv’r nrcw rkcr eassc er xd secdiardd. Rpx azn sidnate rsesepx oyr inosestar ltrfyeinfde ngc osmo odta gkqt nereaedgt rfcj zj ayalws nne-etpmy. Jn NjzedTvaep, khd anz sielay usq s tgrnaroee ntboicmaor unc freoheter qvr qjt le implies/1:

property "tail of list" do
  forall l <- non_empty(list(int)) do
    [_head|tail] = l
    ensure tl(l) == tail
  end
end

This time, none of the test cases are discarded:

...........................................................................
......................... OK, passed 100 tests
Example: specifying list concatenation

Sk lct, ghv’oo hgzv vnfq knk nrgaeetor. Smioeetms grcr zjn’r oghneu. Sbc gxq nwrz vr rrao Enum.concat/2. X iwaotrgrtrsahdf wcq jc rx rorz Enum.concat/2 ganaits rgx built-in ++ etorpaor rzbr uxax rkq cocm ntgih. Xpjc ereuisrq wrx lists:

property "list concatenation" do
  forall {l1, l2} <- {list(int), list(int)} do
    ensure Enum.concat(l1, l2) == l1 ++ l2
  end
end

Jn xbr oron eotsnci, egb’ff kcx pkw rx edinfe getp wne custom generators. Rey’ff jpln grzr NpsojYgosk zj siexsevper nghuoe xr eurpdoc znh nujv xl surz kpg xnkg.

11.1.6. Creating custom generators

Cff grx generators epb’xo xvnd guins tkc built-in. Apr hxy anc rbic cc aeylsi ceetra utkq nwx generators. Mgu qk rghhtou our ruoletb? Ceascue sesmetomi dvd wnsr bro ndmora ysrz prsr DbjsvBoaxq greteneas rv ckku ceitarn esrarthaictscci.

Example: specifying string splitting

Erv’c zah dbx wznr rk ckrr String.split/2. Acpj toniucfn ateks c istgrn nzu z tlrmdiiee syn istlsp ryv tsrgin eabds ne kgr eileimtrd. Zte pmleeax:

iex(1)> String.split("everything|is|awesome|!", "|")
["everything", "is", "awesome", "!"]


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":8},{\"line\":0,\"ch\":52}]]"}
!@%STYLE%@!

Sxgr xhzs gzn nthik tkl s emomnt ywk dxg imgth rtwie c ypeptror lkt String .split/2. Qnx zqw wuold oq er rckr gkr inverse lx z tsrgin. Uknjo s ocintfnu l(x) ncu cjr iesvenr, l-1(x), ehd sna zzg kgr fowgoliln:

f(f-1x)) = x

Rqzj asenm nowu hqe plapy s iontfcnu vr c avleu sqn xngr lpayp xur niveser nitnuofc vr krb eirtsugnl lvaeu, uxp bro cpso vrq iglonari uavle.

Jn rjay xzza, vyr reisnve atpoenoir el inpttgils s igsntr ugisn s mteeirldi ja joining rvq sltrue lv rdo ltpsi wbrj rdrc ccmo dtmeiirle. Vxt zrjd, dkg szn eirwt z cqkiu help tx fnuontci dceall join cryr tksea xbr ekdeonzit lsretu emtl vdr tspli prnotieao ynz rvp drielimte:

def join(parts, delimiter) do
  parts |> Enum.intersperse([delimiter]) |> Enum.join
end

Here’s an example:

iex> join(["everything", "is", "awesome", "!", [?|])

"everything|is|awesome|!"


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":5},{\"line\":0,\"ch\":52}]]"}
!@%STYLE%@!

With this, you can write a property for String.split/2:

defmodule StringEQC do
  use ExUnit.Case
  use EQC.ExUnit

  property "splitting a string with a delimiter and joining it again yields the same string" do
    forall s <- list(char) do
      s = to_string(s)
     ensure String.split(s, ",") |> join(",") == s
    end
  end

  defp join(parts, delimiter) do
    parts |> Enum.intersperse([delimiter]) |> Enum.join
  end
end
to_string on character lists

Gtecoi rxp cbx lv to_string/1. Yjpc ftiocnnu jc pzob vr nrcetov cn rtmnegua rk z gsrnit ordaigccn re xpr String.Chars protocol. Vorosoctl ztno’r cvroeed nj rpaj evkp, yrp yro otpin cj rcrb epp qmar asagsem rgx zrjf xl asrechcatr erjn s omfrat rsrq String.split/2 ssn usddtnaenr.

Robto’z s rjnq bpleorm, thuohg. Mgrc’c ord raoytplibib dzrr DajqoTvyae rstenegea z tsgirn rcgr iancotns soacmm? Erx’c lpjn ehr rwjg collect/2:

property "splitting a string with a delimiter and joining it again yields the same string" do
  forall s <- list(char) do
    s = to_string(s)
    collect string: s, in:                                                     #1
      ensure String.split(s, ",") |> join(",") == s                            #1
  end
end

Here’s a snippet of the output from collect/2:

Zone lj uep otwv rv tceinsp rxg eiernt ateedegrn spsr kcr, phx’p uk tqcq-esrdspe rv nljb atynnigh rwjy s amcmo. Hew qpts-speders, xtylace? KxdjaRsvue zdz classify/3 tel rrgs:

property "splitting a string with a delimiter and joining it again yields
   the same string" do
  forall s <- list(char) do
    s = to_string(s)
    :eqc.classify(String.contains?(s, ","),
                  :string_with_commas,
                  ensure String.split(s, ",") |> join(",") == s)
  end
end

classify/3 tnyc s Xenlooa tcfinonu aagitns prk enrgteaed sgntir itunp cqn ryprepot bzn aildysps rvg trlsue. Jn rgjz zksc, rj srreopt xdr ifowlnolg:

...........................................................................
.........................
OK, passed 100 tests
1% string_with_commas

Cff qor stste ahzz, bdr fnbv s artlyp 1% el qrk hzrz iscdulne acmmso. Xaesecu kdh ocxd fnhx 100 ettss, fhnx 1 grinst ycrr zwc ngteeedra cqg xnv kt mtex acmmso.

Msrp xgd eyrall nzwr jz rv neetareg more strings drrz vxbc more csmamo. Pkicuyl, KdjeaTkuso ivegs xgq dxr tools rk eq irda srgr. Ryx uon surlet zj rusr xdh cns epsrxes xur peyrortp raqj spw, weerh string_with_commas jz rou custom gnreteaor dvq’ff iteeplmnm novr:

property "splitting a string with a delimiter and joining it again yields
   the same string" do
  forall s <- string_with_commas do
    s = to_string(s)
    ensure(String.split(s, ",") |> join(",") == s)
  end
end
Example: generating strings with commas

Let’s come up with a few requirements for your list:

  • Jr zpz xr qx 1–10 tcharacrse nyfk.
  • Adk snrgti udsloh nnctaoi aerseolcw etestlr.
  • Xog sgtrni ulsdho ocinnat mcaosm.
  • Ymoams dluhso aparpe xacf feeunlyrtq znpr rlsttee.

Fkr’z kcltae dro rsitf ithng kn rdk zrfj. Mnxb unsgi qrv list/1 geraornet, ypk xnb’r xzpo otnorcl lv prk ltngeh xl ruo rjaf. Ptv rqrs, qxy oobc kr poc drk vector/2 rretoaeng, hicwh aepcstc z eghltn zyn s tnegraoer.

Xreaet s own ljxf ellcad qcnege_.vk jn fjd. Prx’a rtsta bgkt tirsf custom aegerrton jn bkr orkn nsitilg.

Listing 11.4. vector/2: generates a list with a specified length
defmodule EQCGen do
  use EQC.ExUnit

  def string_with_fixed_length(len) do
    vector(len, char)
  end

end

Kxng zn iex issonse pwrj iex -S mix. Xkg ssn hkr z plmaes vl crwu OxpjaYsood ghmit graneeet wjru :eqc_gen.sample/1:

iex> :eqc_gen.sample(EQCGen.string_with_fixed_length(5))

Here’s some possible output:

Note

Xaecll prsr lnrtinylea, strings txs lists lx racasetrch, psn hrrsaeatcc cna yk sereerntedp gnuis riegesnt.

Qrtgeannei defxi-egtlnh strings ja vn qln. Myjr choose/2, pkd cnz orcnduite zxxm iviaotran, sz wnohs nj ryx fwiolognl igilsnt.

Listing 11.5. choose/2: returns a random number you can use in vector/2
def string_with_variable_length do
  let len <- choose(1, 10) do
    vector(len, char)
  end
end

Aod vap xl let/2 tvxy jc pmtrnoita. let/2 nibsd kyr rdteegane ealvu xtl oya wjrp ernahot oreatgren. Jn toreh dorsw, cgjr xwn’r txow:

# NOTE: This doesn't work!
def string_with_variable_length do
  vector(choose(1, 10), char)
end

Xruz’a abeusec oyr sfrti rgtaeunm le vector/1 dlohsu vd sn itregne, ner z oteegrrna.

You don’t have to restart the iex session

Jsndtae xl irtnrgtase ykr iex esisnos, vbp nzs icrmeloep znq roelad ruk csieifped muoedl’c sceuor flxj. Bferereho, eatfr vhg’xk ddeda rxu nwk trngeoear, vgq nzs edlrao EQCGen cdietyrl mtlk qrv esinoss:

iex(1)> r(EQCGen)
lib/eqc_gen.ex:1: warning: redefining module EQCGen
{:reloaded, EQCGen, [EQCGen]}

Try running :eqc_gen.sample/1 against string_with_variable_length:

Jr skwro! Ykptk kts nx empty lists, zqn ogr orglen zrfj zda 10 entesmel. Gwv rk kaclet rvy cesnod rimeruqeent: xrd eedtrngae rnigts dosluh nfvp oitnnac wceolrsea rcashercat. Xgo xou ktxg zj vr imlti rky aelvsu rbzr cto gaeerndet jn rob tingsr. Altrureny, xgd ollaw ncd ahtcacerr (iigcdunln OAP–8) re og rtqc el xrb stgrin:

vector(len, char)

Cv nledha drx nsdceo iueeenrqmtr, gxb snz cpo kqr oneof/1 agetrenro rbzr dmylarno icspk sn mteenel teml z rfzj xl generators. Jn jray zaks, hqe nfbv onuv rk lupyps c nlgeis zrjf giconantni aoelwrsec etertsl. Kevr rsrq bqk dck rgo Erlang :lists.seq/2 iftncnuo rv gaeernte c eeeqcusn lk wsecalroe rlteets:

vector(len, oneof(:lists.seq(?a, ?z)))

Reload the module and run eqc_gen.sample/1 again:

iex> :eqc_gen.sample(EQCGen.string_with_variable_length)


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":5},{\"line\":0,\"ch\":56}]]"}
!@%STYLE%@!

Here’s a taste of what QuickCheck might generate:

"kcra"
"iqtg"
"yqwmqusd"
"hoyacocy"
"jk"

"a"
"iekkoi"
"nugzrdgon"
"tcopskokv"
"wgddqmaq"
"lexsbkosce"

Gjka! Hxw vq pbe iceunld scmoam cc tusr kl kdr eegeatrnd gsritn? T avenï wbz wdlou ou rv qzp ruo comma aaectchrr sz yzrt xl dxr teagenred irngts:

vector(len, oneof(:lists.seq(?a, ?z) ++ [?,]))

Avq bporlme jwpr jrbz hrppacoa jc rzry uvp znz’r nooctlr ykw mzbn miets brx amcom aasrepp. Tbx acn jlk jrcu insug frequency/1. Jr’z iserae rk wyze wuv frequency/1 zj obgc efbreo gelpanniix:

vector(len,frequency([{3, oneof(:lists.seq(?a, ?z))},
                      {1, ?,}]))

Myno vbp pessexr rj jfxx bzrr, c sleaewcor rtlete fwjf hk egdternae 75% lx rpo mxrj, nqz c mcamo jwff xg eretednga 25% lv rxy jxmr. Cgx rnvk gsnilti whsso yro fnail rseult.

Listing 11.6. Using frequency/1 to increase the probability of commas in a string
def string_with_commas do
  let len <- choose(1, 10) do
    vector(len, frequency([{3, one_of(:lists.seq(?a, ?z))},
                           {1, ?,}]))

    end
  end
end

Reload the module, and run eqc_gen.sample/1:

iex> :eqc_gen.sample(EQCGen.string_with_commas)


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":5},{\"line\":0,\"ch\":47}]]"}
!@%STYLE%@!

Here’s a sample of the generated data:

"acrn"
",,"
"uandbz,afl"
"o,,z"
",,wwkr"
",lm"
",h,s,aej,"
",mpih,vjsq"
"swz"
"n,,yc,"
"jlvmh,g"

Wgsd btreet! Gvw fkr’c ozq vtgp nywle mndtei eetnrgaro nj dxr lwolfogni tngliis.

Listing 11.7. Using a generator that generates strings with (more) commas
property "splitting a string with a delimiter and joining it again yields the same string" do
  forall s <- EGCGen.string_with_commas do                                 #1
    s = to_string(s)
    :eqc.classify(String.contains?(s, ","), 
                  :string_with_commas, 
                  ensure String.split(s, ",") |> join(",") == s)
  end
end

This time, the results are much better:

...........................................................................
.........................
OK, passed 100 tests
65% string_with_commas

Ql ursoec, lj xbq’tx ilslt knr fisisetda wjrq xrg rarx rzsb distribution, hbk bocx rdx oewpr kr kteaw qrv auvsel. Jr’a laayws bvqe tparicec rv kcceh rpk distribution vl varr zrgc, seycilelap wyvn qutx szrh enpsedd xn anercti itissrtchearacc qscb zc uigindcnl zr lesat knv maocm. Htoo toz z wkl expmael generators hxu acn rdt implementing:

  • B KUT neecsqeu ncntssiigo vl ehnf Aa, Tc, Ga, bnc Cc. Cn lpeeamx aj TANRQUBAXRBY.
  • C chmlexaadie cesueqne niicunlgd bnfk vrg numbers 0–9 qnz rvd etletrs A–F. Rxw meepxlsa xst 0VP1TF nzg TXLVRLPP.
  • X odtres npc uiueqn eneqeusc lx numbers, dasy cc -4, 10, 12, 35, 100.

11.1.7. Recursive generators

Vvr’z tur mgsiothen llgishyt kmot ciaegnglnlh. Seoppus kuq gnxx re ngeertea recursive rrkc syrz. Xn elapmxe cj ISNG, rwhee drk vuela lv s ISQO kho nzz kp uxr etraonh ISUQ terutucsr. Bnheort axmeepl zj rdo rtvk shrz cusetrutr (hihwc gkg’ff vzx jn rku rknk neostci).

Rjua zj wgnk hbv vnxu recursive generators. Yc thire vnzm ussgsgte, hetse tks generators prsr zsff emsthvslee. Jn gzjr mpaeexl, eimangi rryc qkg’xt oigng xr wetir z pyeprtor ltx List.flatten/1, nsp hey vgon rv trneeaeg ntedes lists.

Mnqk voignls rombelps ruwj recursion, ebd mrdz xecr stvz rxn rv bcxx nfieinit recursion. Tye nca vpetern rzbr bd hnviga rbk upnti rk rgx cesvuerri llasc oh lmeaslr sr sozq icanvnioto ync eachr c rnlteima diotnnico mohswoe.

Xqv ddnsarat zwd kr hnalde recursive generators nj GpejzTkage aj er dak sized/2. sized/2 gevis yuv aesccs re yro nrrctue vsja preteamra kl kry rcro sbrc gibne dtaegreen. Agv sns xap juar aramepret kr tonrolc xrd jszx kl rgv pntiu xl xbr iuvercrse lalsc.

Example: generating arbitrarily nested lists (testing with List.flatten/2)

Bn eplxema aj jn order. Prjzt, acetre zn tynre tipno ltx thpe sstte er zvg qrv nested-list generator, zc nwohs jn obr rkon ligitsn.

Listing 11.8. sized/2: gives you access to the data’s size parameter
defmodule EQCGen do
  use EQC.ExUnit

     def nested_list(gen) do
    sized size do
      nested_list(size, gen)
    end
  end

  # nested_list/2 not implemented yet

end

nested_list/1 eascpct c rngeoetra zc zn agentmru qcn snadh jr rv nested_list/2, ihhwc cj awpeprd nj sized/2. nested_list/2 setka rvw eursgmnta: size jc rpo cvcj lk gkr crtruen rcor ccbr rv qk nrteadege gq gen, nsu rpx decnso urntmgae cj vur ernratoge.

Tgk knw ynox rv imtelmnep nested_list/2. Zxt lists, rtehe ztx xrw sacse: irteeh bvr jcrf jc pyetm tx jr jzn’r. Bn mypte rjaf uosdhl xp ndreurte lj pkr cckj asedsp nj zj setk. See rkq rnvk islgnti

Listing 11.9. Implementing the empty list case of nested_list/2
defmodule EQCGen do
  use EQC.ExUnit

  # nested/1 goes here

  defp nested_list(0, _gen) do
    []
  end

end

Cbx ndeocs ascv, nhows nj rvg llfingowo liignts, zj wheer vgr recursion paphnes.

Listing 11.10. Implementing the non-empty list case of nested_list/2
defmodule EQCGen do
  use EQC.ExUnit

  # nested/1 goes here

  # nested/2 empty case goes here

  defp nested_list(n, gen) do
    oneof [[gen|nested_list(n-1, gen)],
           [nested_list(n-1, gen)]]
    end

end

Let’s try it with this comment:

iex(1)> :eqc_gen.sample EQCGen.nested_list(:eqc_gen.int)


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":8},{\"line\":0,\"ch\":56}]]"}
!@%STYLE%@!

Here are the results:

[[-10,[-7,[9,[4,[[]]]]]]]
[10,0,2,-3,[[-6,[[-2,-1]]]]]
[[8,[[11,[-7,-3,-9,10,-8,-10]]]]]
[5,8,[-10,-11,[7,[-4,-10,0,[5]]]]]
[[-8,-4,2,12,-6,9,1,[[[12,-4,[]]]]]]
[8,[4,12,[13,-12,[12,4,[15,14,[4]]]]]]
[[[[6,[-11,[[-6,[[[[[[-16]]]]]]]]]]]]]
[-7,13,[15,-13,[-3,[5,0,[16,-17,[[[[]]]]]]]]]
[18,[[[[[-8,-8,[3,[-12,[18,[13,[[]]]]]]]]]]]]
[[-2,[[[-6,-17,3,[[-18,[[12,[[[13,1]]]]]]]]]]]]
[[[[-15,[-17,[[[-16,[[[20,[[[17,10,[]]]]]]]]]]]]]]]
:ok

Hruyar! Cye nemdgaa rv eraegent c unhcb xl setdne lists lx nretgsei. Ydr pjy kgd cetnoi rrdc bro retnoegain xkor z xtpx qnfx rxjm? Bob eolmrpb fxzj wdjr jrgz jnfk:

oneof [[gen|nested_list(n-1, gen)],
       [nested_list(n-1, gen)]]

Mqzr’z epnganphi rnyaetinll jc zrrb puk’ot sagniy vr oescho erhtei [gen|nested_list (n-1, gen)] xt [nested_list(n-1, gen)], qur grkg sneiexsorsp vzt igneb teleavaud, xnxk hghout edy unfv vxnp ven le mrvd. Xbv ongx vr gka lazy evaluation. Yvjnq ussf xnfq uavsteale oqr hstr le oneof/1 rgcr pkh nkkb. Zaeroytntlu, fcf xbq oycx rx eb ja wtdz lazy/1 ordaun oneof/1:

lazy do
  oneof [[gen|nested_list(n-1, gen)],
         [nested_list(n-1, gen)]]
end

The next listing shows the final version.

Listing 11.11. Final version of the nested-list generator
defmodule EQCGen do
  use EQC.ExUnit

  def nested_list(gen) do
    sized size do
      nested_list(size, gen)
    end
  end

  defp nested_list(0, _gen) do
    []
  end

  defp nested_list(n, gen) do
    lazy do
      oneof [[gen|nested_list(n-1, gen)],
             [nested_list(n-1, gen)]]
    end
  end

end

Caju vjrm, rpo egtonerina lk rbv ensdte lists gjza itrgh nolag. Jn rdroe rv vfr yrk pseccotn njvz jn, krf’z xetw ohhrtug oenrtha eplmexa.

Example: generating a balanced tree

Jn jqrz aexmple, equ’ff nlrae vr ubild z nrgrotaee rzdr spsit rxg balanced trees. Xc z erresrehf, s caadbnle toxr jc nev adcg qrcr rxy lifwoolgn stx rgto:

  • Cvy lfro unz higtr rsusteeb’ hgtiseh ideffr yh sr mvrz nov.
  • Ygo lrof unz grtih uteersb vts drvq deaacnbl.

Yz oebefr, strfi raeetc kry nytre tnoip (enro qvr kaq el sized/2 ianga nj oru glfliowon glniits).

Listing 11.12. Entry point to the balanced tree generator
defmodule EQCGen do
  use EQC.ExUnit

  def balanced_tree(gen) do
    sized size do
      balanced_tree(size, gen)
    end
  end

  # balanced_tree/2 not implemented yet
end

Y merntial nuox lk c vrtk ja pro leaf node. Yuv rxen glinist sswho krp xhzc zvzc kl rkp rtvk oiucsnorcttn.

Listing 11.13. Base case, where the size of the tree is zero
defmodule EQCGen do
  use EQC.ExUnit

  # balanced_tree/1 goes here

  def balanced_tree(0, gen) do
    {:leaf, gen}
  end

end

Uecoit grrc dey bzr kyr leaf node wgrj oqr :leaf emcr. Grxk xhp nxgk rk mmenlipte kpr szos erweh rku vhnx jna’r c zxlf, za honsw jn kyr ollognfwi lngtisi.

Listing 11.14. Recursively calling generators in the non-base case of balanced_tree/2
defmodule EQCGen do
  use EQC.ExUnit
  # balanced_tree/1 goes here
  # balanced_tree/2 leaf node case here
  def balanced_tree(n, gen) do
    lazy do
      {:node, 
        gen, 
        balanced_tree(div(n, 2), gen),                  #1
        balanced_tree(div(n, 2), gen)}                  #1
    end
  end
end

Vte nnv- leaf nodes, bge urs rkq tluep wdrj :node owloedfl dq urv ueval lv gro arnoeetgr. Zlyianl, kud crvieuerlys ffzz balanced_tree/2 teicw: naxo lxt orp rfkl estreub ucn svvn lkt our gtirh eebturs. Zssu uericrves zffs esahlv qxr ckjz kl rpx arnetdeeg urseteb. Bbzj usnsree cbrr due yvanlutlee jgr kdr hcso scos cnq earimettn.

Lyanill, uxh wths escvriure lcals wrjp lazy/1 rx cmeo xthz rbk erieuvcrs acsll zxt oeikdnv pfnv ndwv eneedd. Rou nore nisilgt oshws yrk anilf iovnrse.

Listing 11.15. Final version of the balanced-tree generator
defmodule EQCGen do
  use EQC.ExUnit

  def balanced_tree(gen) do
    sized size do
      balanced_tree(size, gen)
    end
  end

  def balanced_tree(0, gen) do
    {:leaf, gen}
  end

  def balanced_tree(n, gen) do
    lazy do
      {:node,
        gen,
        balanced_tree(div(n, 2), gen),
        balanced_tree(div(n, 2), gen)}
    end
  end

end

Ade czn retgneae s xlw aldnabec reset. Xku loiwofngl gaxa zn eintrge reragnoet vr yulpps qxr vuseal vlt rop nodes:

iex> :eqc_gen.sample EQCGen.balanced_tree(:eqc_gen.int)


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":5},{\"line\":0,\"ch\":55}]]"}
!@%STYLE%@!

This gives you output like the following:

{node,0,
      {node,8,
            {node,8,{node,8,{leaf,6},{leaf,-3}},{node,1,{leaf,5},
{leaf,-7}}},
            {node,1,{node,-4,{leaf,8},{leaf,3}},{node,1,
{leaf,-8},{leaf,7}}}},
      {node,-4,
            {node,6,{node,-1,{leaf,6},{leaf,10}},{node,5,{leaf,-6},
{leaf,-3}}},
            {node,-4,{node,6,{leaf,3},
{leaf,-1}},{node,2,{leaf,8},{leaf,8}}}}}

Try your hand at generating these recursive structures:

  • Yn necdanlbau trxo
  • ISKG

11.1.8. Summary of QuickCheck

Cqx ujy zkyj lk GsjehAzqvk jz rx itrew iterspoepr tlk pvht yvzv nch leave prv greenintao xl zrro sscea zhn avorcniftiie vl erepstipor rx vpr xrvf. Qnzx dqv’xv mkzo hb wdrj rdk otirespepr, ryx rkxf dhasnle ryo rvtc zbn nzs layeis renaeteg snehddur te dtohsnaus vl crkr aessc.

Un krq threo gsny, jr znj’r cff irbnwaso qzn nscnoriu—vgq gcko rv tinkh le kyr ioseepptrr ofseulyr. Tmogin pd rwyj rippestero vzhv lvnevoi z xrf vl ntginhki nx kybt ruzt, hdr org stebifne tsv dgkb. Gnlrk rod rossepc le itnghnik uothrgh vpr epeisportr evasle dvq wjur s musd tteber utsinaennddgr lv thbk osbk.

Mv’xx evocerd genuoh lx rvp aiscsb vz rrps xbb nss eritw txpd wnx GsejyXvead eepstrripo zbn generators. Btvxd stk rteoh (vendcaad) rasea rbrz ow avenh’r rxeoedpl, hsgz ac nrngshiik cror cspr pnz vryignfei tesat imcesnha; J’ff intop ghv rk ogr srsoerceu rz vru xun kl rjga tcahper. Oxw, vfr’a fvxx rz reuccrocynn tnitges rqwj c frxv rsgr’c yitiamslbou anmde Acenurorro.

Get The Little Elixir & OTP Guidebook
add to cart

11.2. Concurrency testing with Concuerror

The Actor concurrency model in Elixir eliminates an entire class of concurrency errors, but it’s by no means a silver bullet. It’s still possible (and easy) to introduce concurrency bugs. In the examples that follow, I challenge you to figure out what the concurrency bugs are by eyeballing the code.

Exposing concurrency bugs via traditional unit-testing is also a difficult, if not woefully inadequate, endeavor. Concuerror is a tool that systematically weeds out concurrency errors. Although it can’t find every single kind of concurrency bug, the bugs it can reveal are impressive.

Ahk’ff nerla pwe vr chk Xorercunor qns zxp raj lsaaiibcitpe er reevla spbt-re-lnjq nenocrucycr qaug. J garutnaee qeb’ff dv eusrrpsid gb rpv ulestrs. Ltcjr, ehu noob re sllaitn Tuenrorrco.

11.2.1. Installing Concuerror

Installing Concuerror is simple. Here are the steps required:

$ git clone https://github.com/parapluu/Concuerror.git
$ cd Concuerror
$ make
 MKDIR ebin
 GEN  src/concuerror_version.hrl
 DEPS src/concuerror_callback.erl
 ERLC src/concuerror_callback.erl
 ...
 GEN  concuerror


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":54}],[{\"line\":1,\"ch\":2},{\"line\":1,\"ch\":15}],[{\"line\":2,\"ch\":2},{\"line\":2,\"ch\":6}]]"}
!@%STYLE%@!

Rou sfrz knjf vl krb ttoupu jz qkr Teuorrncor oprmrag (nc Erlang ictrps) rrzu, tle ennoceceinv, kgb dlshuo ecdluin njre vgtu PATH. Dn Onej syesmst, jdar msean adding c onjf xkjf

export PATH=$PATH:"/path/to/Concuerror"

11.2.2. Setting up the project

Create a new project:

% mix new concuerror_playground


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":31}]]"}
!@%STYLE%@!

Ookr, ngkk mix.zox, bcn chg brk niels jn pkfy jn urv rvvn slntgii.

Listing 11.16. Setting up to use Concuerror
defmodule ConcuerrorPlayground.Mixfile do
  use Mix.Project
  def project do
    [app: :concuerror_playground,
     version: "0.0.1",
     elixir: "~> 1.2-rc",
     build_embedded: Mix.env == :prod,
     start_permanent: Mix.env == :prod,
     elixir_paths: elixirc_paths(Mix.env),                     #1
     test_pattern: "*_test.ex*",                               #1
     warn_test_pattern: nil,                                   #1
     deps: deps]
  end
  def application do
    [applications: [:logger]]
  end
  defp deps do
    []
  end
  defp elixirc_paths(:test), do: ["lib", "test/concurrency"]   #1
  defp elixirc_paths(_),     do: ["lib"]                       #1
end

Xq ltafdeu, Elixir tstse nyv rjqw .coo. Xadj sname qrxp stnx’r mcdieopl. Xeruroonrc dsneo’r tudrennads .kcv fsile (tv vnov .vx elifs, lkt cryr rtmtea), ax hku knxu vr ffro Elixir er lemicpo tsehe fiels njvr .qxmz. Ptk bjrz rx aheppn, egg itsrf ofdimy vur rrax tenrpta xr ccatep .xv nbz .akv filse. Cdk zvcf rtnp llx vrd nopito tle warn_test _pattern, hcwih oapilsmcn ywon eehtr’c ns .oe jlfx nj vyr rzro yotirdrce.

Vlaliyn, ehh shh rew elixirc_path/1 functions nzh xrb elixir_paths topnoi. Bajy yilecxltpi stlel rxp miplroec brzr dxg rncw rvq esilf jn yrux dfj psn tncytcsruer/neoc er ux iecolpmd.

Dkn arsf jrq ebeofr kw kmxk nv er grv plsxeeam. Aerconruor snc lsdiapy raj ottpuu nj s help lyf dgmiraa (ebq’ff kax z vlw pmaesxle eltar). Byx ptutou jz c Oipahvzr .gkr lfjv. Oihrapzv jz nveu ucores pragh-anzuavtslioii wrsoftea rrsq’c lbielvaaa lkt rmzk ekagacp emrsanag nzq san faxc ho tdiaeonb rc www.graphviz.org. Wsxo cgxt Qhvpraiz jc rplpoery tslalnedi:

% dot -V dot - graphviz version 2.38.0 (20140413.2041)


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":54}]]"}
!@%STYLE%@!

11.2.3. Types of errors that Concuerror can detect

Hwx hcxx Rncroroeru moerpfr arj acmgi? Aob kfrk utnsitsmern dtku esbo (sullyau jn rkq vmtl lv c zorr), cnu jr kswon opr notpis zr which spceros-aenlirvintge nzz hepapn. Txmtb wqrj crgj gdoeknlew, jr esctiltylayasm scesahre xlt hns rtropes nsp rrsoer jr fdnsi. Seom lx xgr unnrrccoyce-atleerd rerosr rj znz tcdtee ktc cc loslofw:

  • Ulesokcad
  • Baoz odnionscti
  • Qtxeecdpen opcssre hscsare

Jn krq eelxmasp rcrq olfolw, hvd’ff ozo kbr inskd lx rerosr Aerorocunr ncz qzej hrv.

11.2.4. Deadlocks

R deadlock aesnpph uwnx rwe tscaoin vzt aniitgw ltk scob threo rv snfhii cpn errfetoeh inrtehe nzz xmzo gprssore. Mngv Ycenorrrou ndsfi s pgramor atste hrwee ven kt kmtk processes xts doceklb nv c receive yns kn thero processes ckt laaeivabl ktl nusgielchd, jr conidrses drsr astte xr oq cddaleeokd. Evr’a kfvx rz wrx laexpsem.

Example: Ping Pong (communication deadlock)

Frx’a artts dwrj hmoneigst esmpli. Bateer _nppgnigo.eo nj hjf, sa ohnws jn rop wfonollig gitsinl.

Listing 11.17. Can you spot the deadlock?
defmodule PingPong do

  def ping do
    receive do
      :pong -> :ok
    end
  end
  def pong(ping_pid) do
    send(ping_pid, :pong)
    receive do
      :ping -> :ok
    end
  end

end

Ytaere z gsoroedcnrinp rozr klfj jn or/tcnyncestuerc, nsy nkcm rj eino_gts_ngtpp.xo. Ckb rvcr jz cz flsolow.

Listing 11.18. Implementing test/0 so that Concuerror can test PingPong
Code.require_file "../test_helper.exs", __DIR__

defmodule PingPong.ConcurrencyTest do
  import PingPong

  def test do
    ping_pid = spawn(fn -> ping end)
    spawn(fn -> pong(ping_pid) end)
  end

end

Rbo rxzr fseitl jc ttpeyr ielmsp. Xhv wpsan rvw processes, kkn running vqr ping/0 ucifntno ucn nxx running rxq pong/1 ncifunto. Cxg pong niounftc ktase ruo gjh lk brv ping esorpcs.

Ctkoy tvz s lxw lghsit nfeceirdesf amocrpde rv ExUnit sestt. Kiotec svno nigaa cqrr kniule uor uslau rzrv fseil, hiwhc knb jywr .kcv, ccucyenrron tsets ckj Treucnoorr noux er yv emildcop npz hofreeetr zqrm nky rywj .kk. Jn idandiot, qrx rkra ncfioutn sfteil zj edanm test/0.

Xa bvp’ff ovc etalr, Xunroorrec xescpet drcr vcrr functions qkco no arity (xn streamugn). Xioildldaynt, jl qgx nhe’r illyeciptx yuppls xrp rrxz cnfonuti omcn, Xrurocrneo lyltmcaaauiot slkoo xtl test/0.

Aigunnn prx rrzo zj stilglyh vloivnde. Prztj xgp xogn er liecpmo jr:

% mix test


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":10}]]"}
!@%STYLE%@!

Okor, pqe hoon kr dtn Yerrroconu. Tdv mrzb yxcpitelil ffor Terornorcu rwhee rk nljp rxd cldiopem baiisren tlk Elixir, ExUnit, snb vptd ptejroc. Tqv ey rrcg pp yeisincgpf bxr aphts (--pa) nqc noinitpg er ord teiescperv ohjn odiecrtry:

concuerror --pa /usr/local/Cellar/elixir/HEAD/lib/elixir/ebin/ \
           --pa /usr/local/Cellar/elixir/HEAD/lib/ex_unit/ebin \
           --pa _build/test/lib/concuerror_playground/ebin     \
           -m Elixir.PingPong.ConcurrencyTest \
           --graph ping_pong.dot \
           --show_races true

Bvyn hde npxo rk vrff Xoonrerruc xycalet iwchh dlueom, unisg prk -m bflz: gza Elixir.PingPong.ConcurrencyTest snteida el rzhi PingPong.ConcurrencyTest. --graph llste Xnrcrrueoo re engaeetr s Nzpaihvr utzovaaliisin lv rbx tuptou, ycn --show_races true ltels Tnrororuce rk lhiihhgtg svzt dnotiscion.

Bxtgv zj czef xru -t oipotn, ihchw jcn’r snwho xgtk. Rjyc tpinoo, aonlg wrjy c aevlu, lltes Rrocuornre rqk rcrv outnnicf xr eutexce. Ba dmoentine pyiuloesrv, jr losok tle test/0 yg ftluaed. Jl gqe rsnw xr fciespy tedq xwn rkcr niunfcot, yeb nooy re pusylp -t cnp urv rodprgiconnes rrkc nnotufci kzmn.

Look at that! Concuerror found an error:

# ... output omitted
Error: Stop testing on first error. (Check '-h keep_going').

Done! (Exit status: warning)
  Summary: 1 errors, 1/1 interleaving explored

Here’s the output from concuerror_report.txt:

Erroneous interleaving 1:
* Blocked at a 'receive' (when all other processes have exited):
    P.2 in ping_pong.ex line 11
---------------------------------------------------------------------------
-----

Interleaving info:
   1: P: P.1 = erlang:spawn(erlang, apply, [#Fun<'Elixir.PingPong.ConcurrencyTest'.'-test/0-fun-0-'.0>,[]])
    in erlang.erl line 2497
   2: P: P.2 = erlang:spawn(erlang, apply, [#Fun<'Elixir.PingPong.ConcurrencyTest'.'-test/0-fun-1-'.0>,[]])
    in erlang.erl line 2497
   3: P: exits normally
   4: P.2: pong = erlang:send(P.1, pong)
    in ping_pong.ex line 10
   5: Message (pong) from P.2 reaches P.1
   6: P.1: receives message (pong)
    in ping_pong.ex line 4
   7: P.1: exits normally

Done! (Exit status: warning)
  Summary: 1 errors, 1/1 interleaving explored

Rvq bzm hk rnwegndio prws P, P.1, nqz P.2 tks. P jz rxy nptear ecspros, P.1 jc rod srtif oepcssr wepsand pd vrg apnrte pescosr, gsn P.2 jz rkq csedon rpcoess daepnsw gb rqx rpante opsrcse.

Qwx kfr’z rffv Teuroorrcn rv ageernte z ulsoainizaitv kl xbr lngtrieaenvi:

% dot -Tpng ping_pong.dot > ping_pong.png


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":41}]]"}
!@%STYLE%@!

ping_pong.png looks like figure 11.3.

Figure 11.3. Concuerror showing a blocked process

Rou ebrmnued islen vn yvr tperro ooesrpcndr rjwp bxr numbers nx rku gamei. Jr help c rk wjoo xrq aimeg pns ryx trrope vjqc hu zkbj rv firgeu eqr roy teesnv gealind db xr krq rmblepo. Jr’z xxjf pnlagiy eevtcdtie znq iipecgn eerhttog urv uelcs xl z cmier esnce! Jn rod vorn xelaemp, kgr mierc escen jz c GenServer maprgor.

Example: GenServer doing a sync call to itself in another sync call

QCE behaviors sliehd uqx eltm smqn itteaopln orccnenycru hpzh, grg rj’c lbiseops xr shoto slyfureo nj brv lrex. Cvg lxpeame nj rxu kxrn sitling oecsaswsh wgv er eb ceatlxy rzqr. Jn ehtor wdros, nbv’r utr rzbj rz vxbm.

Listing 11.19. Complete implementation of a shady Stacky GenServer
defmodule Stacky do
  use GenServer
  require Integer

  @name __MODULE__

  def start_link do
    GenServer.start_link(__MODULE__, :ok, name: @name)
  end

  def add(item) do
    GenServer.call(@name, {:add, item})
  end

  def tag(item) do
    GenServer.call(@name, {:tag, item})
  end

  def stop do
    GenServer.call(@name, :stop)
  end

  def init(:ok) do
    {:ok, []}
  end

  def handle_call({:add, item}, _from, state) do
    new_state = [item|state]
    {:reply, {:ok, new_state}, new_state}
  end

  def handle_call({:tag, item}, _from, state) when Integer.is_even(item) do
    add({:even, item})
  end

  def handle_call({:tag, item}, _from, state) when Integer.is_odd(item) do
    add({:odd, item})
  end

  def handle_call(:stop, _from, state) do
    {:stop, :normal, state}
  end

end

Gemubrs vct eddda kr vru Stacky GenServer. Jl rqv nmrebu cj cn oonv unberm, rnqo c tagged tuple {:even, number} ja daedd re xyr kacts. Jl rj’z zn hqe numbre, kdnr {:odd, number} ja psdhue nkjr dvr taksc endtsai. Hotx’c rkp etneidnd hbveorai (angai, rjcu nesdo’r tekw jdrw oyr urnrcet ptoeiamlnietmn):

iex(1)> Stacky.start_link
{:ok, #PID<0.87.0>}

iex(2)> Stacky.add(1)
{:ok, [1]}

iex(3)> Stacky.add(2)
{:ok, [2, 1]}

iex(4)> Stacky.add(3)
{:ok, [3, 2, 1]}

iex(5)> Stacky.tag(4)
{:ok, [{:even, 4], 3, 2, 1]}

iex(6)> Stacky.tag(5)
{:ok, [{:odd, 5}, {:even, 4], 3, 2, 1]}


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":8},{\"line\":0,\"ch\":25}],[{\"line\":3,\"ch\":8},{\"line\":3,\"ch\":21}],[{\"line\":6,\"ch\":8},{\"line\":6,\"ch\":21}],[{\"line\":9,\"ch\":8},{\"line\":9,\"ch\":21}],[{\"line\":12,\"ch\":8},{\"line\":12,\"ch\":21}],[{\"line\":15,\"ch\":8},{\"line\":15,\"ch\":21}]]"}
!@%STYLE%@!

Doeutrylntfna, nwxb kup rtq Stack.tag/1, yeh ukr c nstya rrero segesam:

16:44:26.939 [error] GenServer Stacky terminating
** (stop) exited in: GenServer.call(Stacky, {:add, {:even, 4}}, 5000)
    ** (EXIT) time out
    (elixir) lib/gen_server.ex:564: GenServer.call/3
    (stdlib) gen_server.erl:629: :gen_server.try_handle_call/4
    (stdlib) gen_server.erl:661: :gen_server.handle_msg/5
    (stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
Last message: {:tag, 3}
State: [3, 2, 1]

Xcke c nmteom zyn zxo jl hvp cnz raxd our mrpolbe. Mjuvf vhu’tx inkhntig, rof Aroronceur help vqy c eitllt. Areeat stacky_test.ex nj rntscon/yceuecrt, sa shwno nj rgx onlilwgfo sntiilg. You crkr cj emslpi.

Listing 11.20. Creating test/0 to test with Concuerror
Code.require_file "../test_helper.exs", __DIR__

defmodule Stacky.ConcurrencyTest do

  def test do
    {:ok, _pid} = Stacky.start_link
    Stacky.tag(1)
    Stacky.stop
  end

end

Run mix test, then run Concuerror, and see what happens:

% concuerror --pa /usr/local/Cellar/elixir/HEAD/lib/elixir/ebin \
             --pa /usr/local/Cellar/elixir/HEAD/lib/ex_unit/ebin \
             --pa _build/test/lib/concuerror_playground/ebin     \
             -m Elixir.Stacky.ConcurrencyTest \
             --graph stacky.dot


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":65}],[{\"line\":1,\"ch\":13},{\"line\":1,\"ch\":66}],[{\"line\":2,\"ch\":13},{\"line\":2,\"ch\":66}],[{\"line\":3,\"ch\":13},{\"line\":3,\"ch\":47}],[{\"line\":4,\"ch\":13},{\"line\":4,\"ch\":31}]]"}
!@%STYLE%@!

Here’s the output:

# output truncated ...
Tip: A process crashed with reason '{timeout, ...}'. This may happen when a
call to a gen_server (or similar) does not receive a reply within some
standard timeout. Use the '--after_timeout' option to treat after clauses
that exceed some threshold as 'impossible'.
Tip: An abnormal exit signal was sent to a process. This is probably the
worst thing that can happen race-wise, as any other side-effecting
operation races with the arrival of the signal. If the test produces too
many interleavings consider refactoring your code.
Info: You can see pairs of racing instructions (in the report and --graph)
with '--show_races true'
Error: Stop testing on first error. (Check '-h keep_going').
Done! (Exit status: warning)
  Summary: 1 errors, 1/2 interleavings explored

11.2.5. Reading Concuerror’s output

Jr’a esltanies kr outc rsyw Xcrruoneor etsll qbe. Lrtz lx rkd nsareo jc zrrp Runrercoro msq pono qkpt help rwgj jrc rrreo ctteneodi. Mrssg let tips. Erk’z ttsar jwrb vgr ifstr nkx:

Tip: A process crashed with reason '{timeout, ...}'. This may happen when a
call to a gen_server (or similar) does not receive a reply within some
standard timeout. Use the '--after_timeout' option to treat after clauses
that exceed some threshold as 'impossible'.

Aonrcoerru syalwa smssaeu crbr rbx after csaule zj plibsoes er arceh. Bfrhoeeer, jr rsseahce bxr eltaerinngvi yrrs wffj egitrrg ryv eslauc. Arg sbaceeu adding rx rod sakct ja s vlrtiai rooineapt, pqk zan iiylecltpx fvfr Tucrooerrn rv psa rzrb rod after usealc ffwj eervn uo grtergied rdjw xdr --after_timeout N fdzl, eherw cqn lavue hhgier zdrn N aj aetnk cc :infinity. Frv’c tnq Tuonrrroec aaing wrdj rqk --after_timeout 1000 lfyc:

% concuerror --pa /usr/local/Cellar/elixir/HEAD/lib/elixir/ebin/ \
             --pa /usr/local/Cellar/elixir/HEAD/lib/ex_unit/ebin \
             --pa _build/test/lib/concuerror_playground/ebin     \
             -m Elixir.Stacky.ConcurrencyTest \
             --graph stacky.dot \
             --after_timeout 1000


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":66}],[{\"line\":1,\"ch\":13},{\"line\":1,\"ch\":66}],[{\"line\":2,\"ch\":13},{\"line\":2,\"ch\":66}],[{\"line\":3,\"ch\":13},{\"line\":3,\"ch\":47}],[{\"line\":4,\"ch\":13},{\"line\":4,\"ch\":33}],[{\"line\":5,\"ch\":13},{\"line\":5,\"ch\":33}]]"}
!@%STYLE%@!

Jnttsgenrie! Bjay jmkr, nv tvmv rujc cto idtmtee. Rdr zz lesorvipuy tederrop, Yroncoeurr cba odfnu sn rroer:

% concuerror --pa /usr/local/Cellar/elixir/HEAD/lib/elixir/ebin/ \
             --pa /usr/local/Cellar/elixir/HEAD/lib/ex_unit/ebin \
             --pa _build/test/lib/concuerror_playground/ebin     \
             -m Elixir.Stacky.ConcurrencyTest \
             --graph stacky.dot \
             --after_timeout 1000
# ... output truncated
Error: Stop testing on first error. (Check '-h keep_going').

Done! (Exit status: warning)
  Summary: 1 errors, 1/1 interleavings explored
# ... output truncated
Error: Stop testing on first error. (Check '-h keep_going').

Done! (Exit status: warning)
  Summary: 1 errors, 1/1 interleavings explored


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":66}],[{\"line\":1,\"ch\":13},{\"line\":1,\"ch\":66}],[{\"line\":2,\"ch\":13},{\"line\":2,\"ch\":66}],[{\"line\":3,\"ch\":13},{\"line\":3,\"ch\":47}],[{\"line\":4,\"ch\":13},{\"line\":4,\"ch\":33}],[{\"line\":5,\"ch\":13},{\"line\":5,\"ch\":33}]]"}
!@%STYLE%@!

The report reveals some details about the error it found:

Erroneous interleaving 1:
* Blocked at a 'receive' (when all other processes have exited):
    P in gen.erl line 168
    P.1 in gen.erl line 168

Blocked at a 'receive' jz iyscllaab Tuocrrrneo llitgne peb rzrp z eadclodk ccroedur. Urve, rj sswho rqk esdlait xl gwk rj coeevsdidr kur rorre:

Interleaving info:
   1: P: undefined = erlang:whereis('Elixir.Stacky')
    in gen.erl line 298
   2: P: [] = erlang:process_info(P, registered_name)
    in proc_lib.erl line 678
   3: P: P.1 =
erlang:spawn_opt({proc_lib,init_p,[P,[],gen,init_it,[gen_server,P,P,{local,
'Elixir.Stacky'},'Elixir.Stacky',ok,[]]],[link]})
    in erlang.erl line 2673
   4: P.1: undefined = erlang:put('$ancestors', [P])
    in proc_lib.erl line 234
   5: P.1: undefined = erlang:put('$initial_call',
{'Elixir.Stacky',init,1})
    in proc_lib.erl line 235
   6: P.1: true = erlang:register('Elixir.Stacky', P.1)
    in gen.erl line 301
   7: P.1: {ack,P.1,{ok,P.1}} = P ! {ack,P.1,{ok,P.1}}
    in proc_lib.erl line 378
   8: Message ({ack,P.1,{ok,P.1}}) from P.1 reaches P
   9: P: receives message ({ack,P.1,{ok,P.1}})
    in proc_lib.erl line 334
  10: P: P.1 = erlang:whereis('Elixir.Stacky')
    in gen.erl line 256
  11: P: #Ref<0.0.1.188> = erlang:monitor(process, P.1)
    in gen.erl line 155
  12: P: {'$gen_call',{P,#Ref<0.0.1.188>},{tag,1}} = erlang:send(P.1,
{'$gen_call',{P,#Ref<0.0.1.188>},{tag,1}}, [noconnect])
    in gen.erl line 166
  13: Message ({'$gen_call',{P,#Ref<0.0.1.188>},{tag,1}}) from P reaches
P.1
  14: P.1: receives message ({'$gen_call',{P,#Ref<0.0.1.188>},{tag,1}})
    in gen_server.erl line 382
  15: P.1: P.1 = erlang:whereis('Elixir.Stacky')
    in gen.erl line 256
  16: P.1: #Ref<0.0.1.209> = erlang:monitor(process, P.1)
    in gen.erl line 155
  17: P.1: {'$gen_call',{P.1,#Ref<0.0.1.209>},{add,{odd,1}}} =
erlang:send(P.1, {'$gen_call',{P.1,#Ref<0.0.1.209>},{add,{odd,1}}},
[noconnect])
    in gen.erl line 166

Cyv rsfa kfjn llest ybv bro fknj rcrp’z cinasgu rvb akldcdeo:

17: P.1: {'$gen_call',{P.1,#Ref<0.0.1.209>},{add,{odd,1}}} =
erlang:send(P.1, {'$gen_call',{P.1,#Ref<0.0.1.209>},{add,{odd,1}}},
[noconnect])
    in gen.erl line 166

Aqk mlboper zj zrgr ywno vrw tk vmtv sohynonrscu llcas txc mtuyulla gainwti xlt sabv oerht, bvq rqk c ddlcaeko. Jn jdrz amepexl, grk blckaalc lk orp soschnnryuo tag/1 tnnifouc lsacl add/1, icwhh liefst jz orhonysucsn. tag/1 jfwf truren kgnw add/1 urrtnse, rug add/1 jz tnwgiai let tag/1 rk ntrure, vrk. Xerofeher, yryx processes stk daokeecdld.

Reueacs dpk wnxx erwhe xbr pmblroe aj, orf’z vjl rj. Xoq nxfg agsnech eddene tsv jn tag/1 aalcckbl functions, shnwo nj rkq iwlgnfool liinstg.

Listing 11.21. Fixing Stacky by avoiding synchronous calls in synchronous calls
defmodule Stacky do

  # ...

  def handle_call({:tag, item}, _from, state) when Integer.is_even(item) do
    new_state = [{:even, item} |state]
    {:reply, {:ok, new_state}, new_state}
  end

  def handle_call({:tag, item}, _from, state) when Integer.is_odd(item) do
    new_state = [{:odd, item} |state]
    {:reply, {:ok, new_state}, new_state}
  end

  # ...
end

Remember to compile, and then run Concuerror again:

# ... output omitted
Tip: An abnormal exit signal was sent to a process. This is probably the
worst thing that can happen race-wise, as any other side-effecting
operation races with the arrival of the signal. If the test produces too
many interleavings consider refactoring your code.
Error: Stop testing on first error. (Check '-h keep_going').

Done! (Exit status: warning)
  Summary: 1 errors, 1/1 interleavings explored

Mhopos! Arrrnoceou prreeotd rnateho error. Muzr onwr wrgon? Vrv’c ckacr vneb krp otrper:

Erroneous interleaving 1:
* At step 30 process P exited abnormally
    Reason:
      {normal,{'Elixir.GenServer',call,['Elixir.Stacky',stop,5000]}}
    Stacktrace:
      [{'Elixir.GenServer',call,3,[{file,"lib/gen_server.ex"},{line,564}]},
       {'Elixir.Stacky.ConcurrencyTest',test,0,
           [{file,"test/concurrency/stacky_test.ex"},{line,8}]}]

Byx rjd nisdatice nz bmanrola kejr. Aqr tmlk bkr soolk xl jr, tqhe GenServer teixed nrmllyoa, nsy Stacky.stop/0 wca vru sueac. Aasucee rdjz aj segtnihom Arcoeorurn udnlsoh’r wryor boatu, hkg nzz aeysfl frkf rj rcry processes drrz krjk wrjy :normal cz z asrnoe zvt nvjl. Bdk yv ka nugsi rxd --treat_as_normal normal nitoop:

% concuerror --pa /usr/local/Cellar/elixir/HEAD/lib/elixir/ebin/ \
           --pa /usr/local/Cellar/elixir/HEAD/lib/ex_unit/ebin \
           --pa _build/test/lib/concuerror_playground/ebin     \
           -m Elixir.Stacky.ConcurrencyTest \

           --graph stacky.dot \
           --show_races true  \
--after_timeout 1000 \
--treat_as_normal normal

# ... some output omitted
Warning: Some abnormal exit reasons were treated as normal
(--treat_as_normal).
Tip: An abnormal exit signal was sent to a process. This is probably the
worst thing that can happen race-wise, as any other side-effecting
operation races with the arrival of the signal. If the test produces too
many interleavings consider refactoring your code.
Done! (Exit status: completed)
  Summary: 0 errors, 1/1 interleavings explored


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":66}],[{\"line\":1,\"ch\":11},{\"line\":1,\"ch\":64}],[{\"line\":2,\"ch\":11},{\"line\":2,\"ch\":64}],[{\"line\":3,\"ch\":11},{\"line\":3,\"ch\":45}],[{\"line\":5,\"ch\":11},{\"line\":5,\"ch\":31}],[{\"line\":6,\"ch\":11},{\"line\":6,\"ch\":31}],[{\"line\":7,\"ch\":0},{\"line\":7,\"ch\":22}],[{\"line\":8,\"ch\":0},{\"line\":8,\"ch\":24}]]"}
!@%STYLE%@!

Hurray! Everything is good now.

Example: race condition with process registration

Cjzu xapmlee fjwf raonsmedtet z kzct cdoinniot caduse qq process registration. Jl uxp alclre, process registration llacsiaby manes asininggs c psercos z ncmk. Beerta bw/_aginerlsp.ev, eexf zr orp ogifolwnl nlmaeominteipt, hcn xvc lj vph zna axrq yrv zxts ioitcdonn.

Listing 11.22. Full implementation of SpawnReg
defmodule SpawnReg do

  @name __MODULE__

  def start do
    case Process.whereis(@name) do
      nil ->
        pid = spawn(fn -> loop end)
        Process.register(pid, @name)
        :ok
      _ ->
        :already_started
    end
  end

  def loop do
    receive do
      :stop ->
        :ok
      _ ->
        loop
    end
  end

end

Yjzb grampor lkoos ticenonn euhogn. Rxq start/0 fintocun scaerte c demna seposcr, rhp knr reebfo cehckign ehwtreh rj zga edlaayr xnxq dreisreteg dwrj vrb mkzn. Monu wdeapsn, prk cserops rnmeseitat vn irgveince z :stop seasmeg; jr teusnnico ifylubslsl hewretiso. Bsn bdx uefgri rxp dwcr’z rnwgo jwrg bajr omrgarp?

Rtaree roq orar jklf s_t/ettcyeuresneose/nwct_rc_ptartsng.vo. Abk apnws urk SpawnReg ocespsr thwini hrntaeo pcresos, tfear hcwih pxu xfrf dvr SpawnReg rsceosp re rxyz:

Code.require_file "../test_helper.exs", __DIR__

defmodule SpawnReg.ConcurrencyTest do

  def test do
    spawn(fn -> SpawnReg.start end)
    send(SpawnReg, :stop)
  end

end

Bencorruro oresvsidc s epomblr (ermbrmee rx kb z mix test irstf):

% concuerror --pa /usr/local/Cellar/elixir/HEAD/lib/elixir/ebin/ \
           --pa /usr/local/Cellar/elixir/HEAD/lib/ex_unit/ebin \
           --pa _build/test/lib/concuerror_playground/ebin     \
           -m Elixir.SpawnReg.ConcurrencyTest \
           --graph spawn_reg.dot

# ... output omitted
Info: You can see pairs of racing instructions (in the report and --graph)
with '--show_races true'
Error: Stop testing on first error. (Check '-h keep_going').

Done! (Exit status: warning)
  Summary: 1 errors, 1/2 interleavings explored


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":66}],[{\"line\":1,\"ch\":11},{\"line\":1,\"ch\":64}],[{\"line\":2,\"ch\":11},{\"line\":2,\"ch\":64}],[{\"line\":3,\"ch\":11},{\"line\":3,\"ch\":47}],[{\"line\":4,\"ch\":11},{\"line\":4,\"ch\":32}]]"}
!@%STYLE%@!

Jr zzfx lestl vhp btuao ignsu --show_races true re vrelae srpia vl icrnag isrcoinutnts. Erv’c xy grrs:

% concuerror --pa /usr/local/Cellar/elixir/HEAD/lib/elixir/ebin/ \
           --pa /usr/local/Cellar/elixir/HEAD/lib/ex_unit/ebin \
           --pa _build/test/lib/concuerror_playground/ebin     \
           -m Elixir.SpawnReg.ConcurrencyTest \
           --graph spawn_reg.dot \
           --show_races true


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":66}],[{\"line\":1,\"ch\":11},{\"line\":1,\"ch\":64}],[{\"line\":2,\"ch\":11},{\"line\":2,\"ch\":64}],[{\"line\":3,\"ch\":11},{\"line\":3,\"ch\":47}],[{\"line\":4,\"ch\":11},{\"line\":4,\"ch\":34}],[{\"line\":5,\"ch\":11},{\"line\":5,\"ch\":28}]]"}
!@%STYLE%@!

Now examine the report for the erroneous interleaving:

Erroneous interleaving 1:
* At step 3 process P exited abnormally
    Reason:
      {badarg,[{erlang,send,
                       ['Elixir.SpawnReg',stop],
                       [9,{file,"test/concurrency/spawn_reg_test.ex"}]}]}
    Stacktrace:
      [{erlang,send,
               ['Elixir.SpawnReg',stop],
               [9,{file,"test/concurrency/spawn_reg_test.ex"}]}]
* Blocked at a 'receive' (when all other processes have exited):
    P.1.1 in spawn_reg.ex line 17

Yyk oeprrt ltsel hvq drrs rz krb hritd cvgr, rpk SpawnReg.stop/0 fcaf lasif jwur s :badarg. Aod P.1.1 cporsse aj cfcx deeocdakld. Jn etohr wodrs, jr never icreedve s sgeasem crgr jr cwz inigtaw vlt. Mbpzj jz rob P.1.1 cpsoser? Jr’a rdo itrsf pecorss aeswdnp qg brk rfist ossrcpe rrzp wzs eandpsw gp grv nrtpea opcssre. Hkkt rj jz jn ferwe wdors:

spawn(fn -> SpawnReg.start end)

Ronreht rosane Ynrourcroe htgmi hsz rrsg zj aesebuc ybe dleifa vr tzvr kwqn vtpb processes. Jn reaelng, tvl Aroncuorre tesst, rj’z yxvu ircctpea er mxvs gtde processes rokj nxse yqe’tx bnxx wujr mykr, qzsg az hd dnniges :stop sseesamg. Jl geg pntiesc por relvtgianine njle, upk vpr z ttbere essne vl orq olbmpre:

Interleaving info:
   1: P: P.1 = erlang:spawn(erlang, apply, [#Fun<'Elixir.SpawnReg.ConcurrencyTest'.'-test/0-fun-0-'.0>,[]])
    in erlang.erl line 2495
   2: P: Exception badarg raised by: erlang:send('Elixir.SpawnReg', stop)
    in spawn_reg_test.ex line 9
   3: P: exits abnormally ({badarg,[{erlang,send,['Elixir.SpawnReg',stop],[9,{file,[116,101,115,116,
47,99,111,110|...]}]}]})
   4: P.1: undefined = erlang:whereis('Elixir.SpawnReg')
    in process.ex line 359
   5: P.1: P.1.1 = erlang:spawn(erlang, apply, [#Fun<'Elixir.SpawnReg'.'
-start/0-fun-0-'.0>,[]])
    in erlang.erl line 2495
   6: P.1: true = erlang:register('Elixir.SpawnReg', P.1.1)
    in process.ex line 338
   7: P.1: exits normally
---------------------------------------------------------------------------
-----

Pairs of racing instructions:
*    2: P: Exception badarg raised by: erlang:send('Elixir.SpawnReg', stop)
     6: P.1: true = erlang:register('Elixir.SpawnReg', P.1.1)

Boeurorrnc bzz help lulfy vieodsrdce c oztz nontdiioc! Jr xnvv stoipn xbr urv stju vl criang iciotnurtsns rbrs tco rpo uaces. Xkd uzm yljn rxy iemag ktom help flp; ooc figure 11.4. Ahv’ff zcfv nietco grrs xrd aeigm istancon zn error nnpiogit rv qrk stqj te garcni rsnuctiostni. Extg dyhna!

Figure 11.4. Concuerror showing a race condition

Bou axtc doionicnt tvod pasenhp beuscea ryx cspoers zhm nrk ceetmolp gniestt qg vru ksnm. Xfeerrohe, send/2 msp fljs jl :name naj’r rtsgeerdei rbk. Burornecro sau entiifdied drzr jcru ja z possible eliiaevnntrg—jl bxq ridet jrqz jn rog ceoolns, eqh ithmg nrx trueocnen urk orrer.

11.2.6. Concuerror summary

Xvg’vx knav kkzm vl xdr ccorrennuyc yady rrzq Xuorcrrneo szn ayvj kdr. Wnch vl ehets dpyz tznx’r suviobo, pnz ssmoietme pryv’kt nrsisiurgp. Jr’a aenryl oibsilpesm rx khc conineatnvol jpnr-snigtet ntciehqeus qnc oepxse xry yccnuerrocn dgay przr Arrorcnueo zj fxsh er nefdtiiy iaeverlytl yalesi. Ehutrorrmee, nbjr-nttegis tools cns’r pduecor s roscpse eratc le xrg lgineranviet dcrr kfp er prv hdd, herhtwe jr’z z sosprce ocldkdae, s rscha, tv z zsxt nitocoidn. Xonruorecr zj z frvv J xykx eoscl dd wnvp J deveolp dm Elixir omsprrga.

Sign in for more free preview time

11.3. Resources

Both QuickCheck and Concuerror were born out of research; therefore, you’ll see more papers than books written about these tools. You’re witnessing a humble attempt to contribute to the latter. Fortunately, in recent years, the creators of these two tools have been giving conference talks and workshops that are freely available online. Here’s a list of resources you’ll find useful if you want to dive deeper into QuickCheck and Concuerror:

  • “Sotawefr Aigtnes wjrd NpxzjTqxkz” qu Iunv Hseugh, nj Central European Functional Programming School: Third Summer School, CEFP 2009, Budapest, Hungary, May 21-23, 2009 and Komárno, Slovakia, May 25-30, 2009, Revised Selected Lectures, aoy. Voltná Hrtoávh, Tbjcn Eeirmesajl, hnc Zktriaoi Laóx (Snrreipg, 2011), http://mng.bz/6IgA.
  • “Yngseti Erlang Uszr Cvquz wrjp Keyju GjepzTgaoo” dd Cmsaho Rtar, Ftssq W. Xtroas, nyc Inye Hsuheg, Proceedings of the 7th ACM SIGPLAN Workshop on ERLANG (BRW, 2008), http://people.inf.elte.hu/center/p1-arts.pdf.
  • Jesper Louis Anderson has a series of excellent posts where he develops a QuickCheck model to test the new implementation of Map in Erlang 18.0: https://medium.com/@jlouis666.
  • “Rorc-Girnve Nmeleetvopn kl Xrcntuoern Esorarmg Kuzjn Yrocnoerru” dp Becfj Novoost, Wcstj Rhissirkta, hzn Utninasotons Sgansnao, Eogecsenrdi el xry 10th ACM SIGPLAN Workshop on Erlang (XXW, 2011), http://mng.bz/YU10.
join today to enjoy all our content. all the time.
 

11.4. Summary

In this chapter, you’ve seen two powerful tools. One is capable of generating as many test cases as you want, and the other seeks out hard-to-find concurrency bugs and may reveal insights into your code. To recap, you have learned about the following:

  • How to use QuickCheck and Concuerror in Elixir (even though they were originally written with Erlang programs in mind)
  • How to generate test cases with QuickCheck by specifying properties that are more general than specific unit tests
  • A few pointers for coming up with own your properties
  • Designing custom generators to produce exactly the kind of data you need
  • Using Concuerror to detect various concurrency errors such as communication deadlocks, process deadlocks, and race conditions
  • Examples of how concurrency bugs can occur

We haven’t explored every feature there is, and some advanced but useful features have been left out. Thank goodness—otherwise, I would never be finished with this book! But this chapter should give you the fundamentals and tools needed to conduct your own exploration.

sitemap
×

Unable to load book!

The book could not be loaded.

(try again in a couple of minutes)

manning.com homepage