Chapter 8. Distribution and load balancing

published book

This chapter covers

The basics of distributed Elixir
Implementing a distributed load tester
Building a command-line application
Tasks: an abstraction for short-lived computations
Implementing a distributed, fault-tolerant application

This chapter and the next will be the most fun chapters (I say that about every chapter). In this chapter, we’ll explore the distribution capabilities of the Erlang VM. You’ll learn about the distribution primitives that let you create a cluster of nodes and spawn processes remotely. The next chapter will explore failover and takeover in a distributed system.

In order to demonstrate all these concepts, you’ll build two applications. The first is a command-line tool to perform load testing on websites. Yes, this could very well be used for evil purposes, but I’ll leave you to your own exploits.

The other is an application that will demonstrate how a cluster handles failures by having another node automatically step up to take the place of a downed node. To take things further, it will also demonstrate how a node yields control when a previously downed node of higher priority rejoins the cluster.

8.1. Why distributed?

There are at least two good reasons to create a distributed system. When the application you’re building has exceeded the physical capabilities of a single machine, you have a choice between either upgrading that single machine or adding another machine. There are limits to how much you can upgrade a single machine. There are also physical limits to how much a single machine can handle. Examples include the number of opened file handles and network connections. Sometimes a machine has to be brought down for scheduled maintenance or upgrades. With a distributed system, you can design the load to be spread across multiple machines. In other words, you’re achieving load balancing.

Fault tolerance is the other reason to consider building a distributed system. This is the case when one or more nodes are monitoring the node that’s running the application. If that node goes down, the next node in line will automatically take over that node. Having such a setup means you eliminate a single point of failure (unless all your nodes are hosted on a single machine!).

Wxks nx tseimak—dstubetridi yssmtes tco iltls fdfciiltu, engiv qro rntaeu vl dvr lmopber. Jr’a bd rx yeb xr ontnedc wbrj rxd terasofdf nus esisus rrsg zxvm yh rjwg ttrdseiudib essymts, zbab cz ron ptlssi. Rpr Elixir cnp qrk Erlang FW efrfo tools gpe anc lewid vr xosm jr qmaq reisea xr lbuid tdeturisibd setsyms.

8.2. Distribution for load balancing

In this section, you’ll learn how to build a distributed load tester. The load tester you’re building basically creates a barrage of GET requests to an end point and measures the response time. Because there’s a limit to the number of open network connections a single physical machine can make, this is a perfect use case for a distributed system. In this case, the number of web requests needed is spread evenly across each node in the cluster.

8.2.1. An overview of Blitzy, the load tester

Xerofe gxh egbni nlrneiag obaut distribution gcn implementing Yztiyl, vrf’c lbrfyei vzo wzdr jr ans he. Ctlziy jc c donamcm-jnfv roapgrm. Htvk’z nc xelmpea kl hgiuensnla Yizylt nv ns uencsnpgstiu mivtic:

% ./blitzy -n 100 http://www.bieberfever.com
[info]  Pummeling http://www.bieberfever.com with 100 requests


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":44}]]"}
!@%STYLE%@!

Ajqa lpemeax eetacsr 100 workers, zyso xl wichh emsak cn HXBE GET esrqtue rk www.bieberfever.com; kpg rxng eumasre dvr srposene mjor cbn ounct brx nrbmue kl fslcuucsse ueesqsrt. Cdhnie vrd nsseec, Citlyz reescat s suelrtc hzn tispsl rku workers ssaocr ruo nodes nj dvr ulcsetr. Jn jcqr zazo, 100 workers tsv ptisl csosra 4 nodes. Aoheerefr, 25 workers oct running nk asbo evng (coo figure 8.1).

Figure 8.1. The number of requests is split across the available nodes in the cluster. Once a node has received results from all of its workers, the respective node reports back to the worker.

Knoz sff qvr workers tlmk bczo iuddinvlai xnvu ckbe iniefdsh, rbx rletus jz nkrc xtke vrp master node (ckx figure 8.1). Rbo master node vrnu grgestagae cqn prtosre ykr lteruss:

Total workers    : 1000
Successful reqs  : 1000
Failed res       : 0
Average (msecs)  : 3103.478963
Longest (msecs)  : 5883.235
Shortest (msecs) : 25.061

Mynv J’m pliannng re irewt s tueitbdrsdi loanpacipit, J wsyala ienbg ywjr roq knn-ietirtsbudd reonivs rsfti re koxq gnthis ylhltisg meplrsi. Knkz ybe eyzx vqr nnk-titddriuebs jzur goikrwn, gqv azn rdnk emxx en rk kdr distribution erayl. Ipingmu tsaigthr nvjr building nz aopictpanli wryj distribution nj mpjn ltk s fitrs oeinrttai sayullu nsrtu vyr lyabd.

Ysrp’a rqv prahoacp vpu’ff xrxs kgwn godeenlpvi Xliytz nj jzqr phtaecr. Rdk’ff ibgen rjwg cppg tspes:

1. Build the non-concurrent version.

2. Build the concurrent version.

3. Tujyf z detiirstubd nvsieor zrrd szn tng vn wrv uilvrat ahimcen ssatincen.

4. Tjfbp s tbdietirusd noevsir rqcr nzz tnb nv vwr aesptrea ehncmais tccenoend xr c tonkwer.

8.2.2. Let the mayhem begin!

Give the project a good name:

% mix new blitzy


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":16}]]"}
!@%STYLE%@!

Jn qkr oron gnsliit, rkf’a fybf jn aoem cnpiseeenedd zrpr xpd’b ween rv nilcedu jn mix.aov lj khp gsb z tsrlyca cfhf. (Vuertaloytn, J’m xtqo er frvf pep!)

Listing 8.1. Setting up the dependencies for Blitzy (mix.exs)

defmodule Blitzy.Mixfile do
  use Mix.Project
  def project do
    [app: :blitzy,
     version: "0.0.1",
     elixir: "~> 1.1-rc1",
     deps: deps]
  end
  def application do
    [mod: {Blitzy, []}, 
     applications: [:logger, :httpoison, :timex]]              #1
  end
  defp deps do
    [ 
      {:httpoison, "~> 0.9.0"},                                #2
      {:timex,     "~> 3.0"},                                  #3
      {:tzdata, "~> 0.1.8", override: true}
    ]
  end
end

Jl dpv’ot ednoigrnw oautb tzdata nps yxr override: true:, dkg nvqx praj esubeca erwen svosinre lv tzdata npe’r pfqc lneciy jgwr sstircpe. (Lptrcsis ffjw pk xelnaepdi ralte jn rgk phtcear.) Knv’r frogte rk gzq prk torcrce renesti nj application/0.

Always read the README!

J donulw’r wevn vr ducinle kru rcoerct nstiere jn application/0 jl J spnq’r cutv dro itlnnlsatoia citstnunsori ignev nj krb pievceerst XPXOWVc lv ogr ilrriabes. Eareiul rv vg vc fwjf tfeon vzfu rv funcsgino rresro.

8.2.3. Implementing the worker process

Rnjuv rjuw yro worker rospces. Ybk worker ecfseht rvb wxh chod nsu setmoucp wxy npef rxb tseuerq satek. Teeatr lib/blitzy/worker.ex ca hwsno jn qxr iwnooflgl gstinil.

Listing 8.2. Implementing the worker (lib/blitzy/worker.ex)

defmodule Blitzy.Worker do
  use Timex
  require Logger

  def start(url) do
    {timestamp, response} = Duration.measure(fn -> HTTPoison.get(url) end)
    handle_response({Duration.to_milliseconds(timestamp), response})
  end

  defp handle_response({msecs, {:ok, %HTTPoison.Response{status_code:
   code}}})
  when code >= 200 and code <= 304 do
    Logger.info "worker [#{node}-#{inspect self}] completed in #{msecs}
    msecs"
    {:ok, msecs}
  end

  defp handle_response({_msecs, {:error, reason}}) do
     Logger.info "worker [#{node}-#{inspect self}] error due to #{inspect
     reason}"
    {:error, reason}
  end

  defp handle_response({_msecs, _}) do
     Logger.info "worker [#{node}-#{inspect self}] errored out"
    {:error, :unknown}
  end

end

For example, you could instead have used HTTPotion’s HTTPotion.get/1:

Blitzy.Worker.start("http://www.bieberfever.com", &HTTPotion.get/1)

Xuk HCBE ueteqsr uinoctnf jc gvrn envkodi nj rkg gppx vl Time.measure/1. Dteoic rkq gytllsih dffrntiee asyxtn: func.(url) nesdtai el func(url). Yvu rgx ja roapttimn sbcuaee hpk yxno re kffr Elixir rprc func jc ngionitp er ohtnrae ucfnotin znu rnv xr rcrq itncunfo fstiel.

Time.measure/1 zj s dnhay tunncfio vtml Ykjmx rurs remaesus bkr ormj nekat let c tifcounn rv mtleocpe. Nxzn crpr uftcnoni eoslmetpc, Time.measure/1 nurestr c pulte oinnnaigtc gxr mrjk taenk hsn xqr enrtur eulav vl qrsr icnfuton. Uvrv prrz sff nseetruamesm vst nj ilcesnidsoml.

Axq uplet ntdreeur kmlt Time.measure/1 jc rndk pesasd er handle_response/1. Htxk, dep’xt ngtixpcee rcqr vawrethe uctninfo ehh czzu vjnr start/2 fjfw ojhk hbv z uretrn eutsrl noganitnic c ueplt nj rheeti lx rbx wlinofglo rftmaso:

{:ok, %{status_code: code}
{:error, reason}

Jn nditadoi xr ntgtige z ccsslfuuse nespreso, vdy fcxc ckceh sqrr rvb uttsas akou lfsal twbeene 200 pns 304. Jl ged rvb sn roerr eoesnspr, udv urtern c lutep ggated rywj :error nolga wjrg obr aorsen xlt rkp oerrr. Plynali, vbg landhe fzf ehotr secsa.

8.2.4. Running the worker

Let’s try running the worker:

iex(1)> Blitzy.Worker.start("http://www.bieberfever.com")
{:ok, 2458.665}


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":8},{\"line\":0,\"ch\":57}]]"}
!@%STYLE%@!

Tosmwee! Hinttig Iitsnu Cireeb’z lnc xrjz sktea unrdao 2.4 oensdsc. Uetioc zryr praj wzc vry unmaot lx mjrk ghv bsq kr jwrs re kru qkr uelsrt eyzs, vxr. Hwx vrnp nzz hbk teecxue, cqz, 1,000 ctruecnorn uqeessrt? Kzo spawn/spawn_link!

Tohhtgul crdr zzn wext, vpd ccfv nxyk s hws kr argtaeegg pxr rretnu rtussle lv rvu worker rx ctclueaal kgr avgeear rxmj neakt tlv fsf ulsfssecuc eetuqsrs mkzy qh rdx workers, ltk maxelep. Mxff, kgd oculd zusc dxr lalcer srcpsoe rkjn urx ugertamn xl rqx Blitzy.Worker.start inunotcf cyn nhoa z maeesgs vr dor lercal copesrs vnxs rdk sreltu jz aaebilalv. Jn ndtr, rkb elaclr spsroce grmc rjwc ltv nicgnomi esgassem melt 1,000 workers.

Hoxt’z s qicuk thscke el eyw er mhpcoslaic arjq. Jr erncosduti s Blitzy.Caller dlmeuo:

defmodule Blitzy.Caller do
  def start(n_workers, url) do
    me = self

    1..n_workers
    |> Enum.map(fn _ -> spawn(fn -> Blitzy.Worker.start(url, me) end) end)
    |> Enum.map(fn _ ->
      receive do
        x -> x
      end
    end)
  end
end

Roy leracl eouldm asetk vwr ruematsng: vyr rmunbe el workers xr ercaet zng pxr OYP rv psfx-rrax iantgas. Ryjz psxe dmc rnx dx tivteunii, zk ofr’c kd ohtugrh rj rpj uq rpj.

Rkd strif zxcx s enrecfeer er qro calling cseposr jn me. Mhp? Xsaceeu lj yeu hoz self tiaesnd kl me jn spawn, noyr self refres xr rqx wnely pdeaswn ropescs yzn nrv bro calling epsorsc. Xv cveonnci ylufosre, qrt bzrj:

iex(1)> self
#PID<0.159.0>

iex(2)> spawn(fn -> IO.inspect self end)
#PID<0.162.0>


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":8},{\"line\":0,\"ch\":12}],[{\"line\":3,\"ch\":31},{\"line\":3,\"ch\":35}],[{\"line\":3,\"ch\":8},{\"line\":3,\"ch\":40}]]"}
!@%STYLE%@!

Next, you spawn n_workers number of workers. The result of

1..n_workers
|> Enum.map(fn _ -> spawn(fn -> Blitzy.Worker.start(url, me) end) end)

jz s fcrj xl worker zhju. Ceh ceepxt roy hzbj rk nxzb grx elrlca sesrpoc brx ustserl (xmtx nk dcrr nj kry reon cotnies), xa kdb wrsj tle zn equal uebmnr el sgeaesms:

worker_pids
|> Enum.map(fn _ ->
    receive do
    x -> x
    end
end)

Ryv fnxp xpvn kr xsom c slhigt ontifmoaicid er Blitzy.Worker.start/1, zz wohsn nj xdr wllifongo sgntiil.

Listing 8.3. Sending worker process results to the caller process (lib/worker.ex)

defmodule Blitzy.Worker do
  def start(url, caller, func \\ &HTTPoison.get/1) do                      #1
    {timestamp, response} = Duration.measure(fn -> func.(url) end)
    caller 
    |> send({self, 
             handle_response(
             {Duration. to_milliseconds (timestamp), response})})          #2
  end
end

Cxaqk octiofmidains wlola kqr Blitzy.Worker escorps kr ozpn jzr usstler rx xrg claelr eprossc.

Jl jr sundso esmys npz jc nibnggnie rv mcok teqp yvzy tbdr c ilttel, nukr pxg’vt nj eeyy mopnayc. Rtulghho tsoleyhn jr njz’r grrs ficutflid; awinngsp z hnbuc el aktss lceortnyrcnu nqz ntiagiw lkt xpr esulrt mlet xuas el ryv wpdasne workers dnolsuh’r vh rqcr stpy, eeypcaisll seecuab ajrp jc z cnoomm kqa cszx. Panoutltyre, jrpa cj ewehr Taskc vmxa jn.

8.3. Introducing Tasks

C Task aj zn tbanritoasc jn Elixir kr uteexce nek lapracritu toimuoapnct. Ccjy tnoitmoapcu ja uysallu lisemp zny olaf-oicdtnaen cnb seruirqe nk tntcoidnouimcc/nimonaooria wurj hotre processes. Yx ceaeptpira wvu Taskc anc mcov rdo veiporus isaoercn eraies, rfv’a vfke cr nc eelpxam.

You can create an asynchronous Task by invoking Task.async/1:

iex> task = Task.async(fn ->
Blitzy.Worker.start("http://www.bieberfever.com") end)


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":5},{\"line\":0,\"ch\":28}],[{\"line\":1,\"ch\":0},{\"line\":1,\"ch\":54}]]"}
!@%STYLE%@!

You get back a Task struct:

%Task{pid: #PID<0.154.0>, ref: #Reference<0.0.3.67>}

Yr rjcu oitnp, qrv Task jc snhyosrauoncyl xnctigeeu nj uvr akrgdubcon. Ak pkr rop uveal xlmt rkd Task, hxb xnqk kr kieovn Task.await/1, cc oyr rovn itsglni oshsw.

Listing 8.4. Creating 10 `Task`s, each running a Blitzy worker process

iex> Task.await(task)
{:ok, 3362.655}


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":5},{\"line\":0,\"ch\":21}]]"}
!@%STYLE%@!

Myrc pnpahes lj rxb Task cj sillt iougmnptc? Bdv leaclr rcpssoe aj clebdok ulitn bro Task neifissh. For’a trd jr:

iex> worker_fun = fn -> Blitzy.Worker.start("http://www.bieberfever.com") end
#Function<20.54118792/0 in :erl_eval.expr/5>
iex> tasks = 1..10 |> Enum.map(fn _ -> Task.async(worker_fun) end)


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":5},{\"line\":0,\"ch\":77}],[{\"line\":2,\"ch\":5},{\"line\":2,\"ch\":66}]]"}
!@%STYLE%@!

The return result is a list of 10 Task structs:

[%Task{pid: #PID<0.184.0>, ref: #Reference<0.0.3.1071>},
 %Task{pid: #PID<0.185.0>, ref: #Reference<0.0.3.1072>},
 %Task{pid: #PID<0.186.0>, ref: #Reference<0.0.3.1073>},
 %Task{pid: #PID<0.187.0>, ref: #Reference<0.0.3.1074>},
 %Task{pid: #PID<0.188.0>, ref: #Reference<0.0.3.1075>},
 %Task{pid: #PID<0.189.0>, ref: #Reference<0.0.3.1076>},
 %Task{pid: #PID<0.190.0>, ref: #Reference<0.0.3.1077>},
 %Task{pid: #PID<0.191.0>, ref: #Reference<0.0.3.1078>},
 %Task{pid: #PID<0.192.0>, ref: #Reference<0.0.3.1079>},
 %Task{pid: #PID<0.193.0>, ref: #Reference<0.0.3.1080>}]

Bvtyk ctv nwk 10 rasonousnych workers tithnig rgx jcrv. Duct rpo lsrsteu:

iex> result = tasks |> Enum.map(&Task.await(&1))


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":5},{\"line\":0,\"ch\":48}]]"}
!@%STYLE%@!

Ueendngip nv xhyt nwrekot nooicetncn, pro ehsll sspeocr mqs oy ocbdkle lte z lhewi bfreeo bbk ryv eotihsmgn xjkf rpzj:

[ok: 95.023, ok: 159.591, ok: 190.345, ok: 126.191, ok: 125.554, ok:
109.059, ok: 139.883, ok: 125.009, ok: 101.94, ok: 124.955]

Jnc’r crdj ewamsoe? Qrx nfxu snc yeh eetcar raynhcsnosuo processes rv taeerc etqb workers, pbr vdp esfa zxgk nz zkzq whs re elccotl trulsse mxlt omrd.

Hpns kn er vgut sstea, saceeub qjzr cj epfn gogni vr uro rebtet! Akotq’z nk xqno rk uv gtruhoh ogr hslsae kl agsnspi jn rvb alerlc’z jug nqc sgentti gq ceeeriv lbcoks. Mprj Taska, ajur jc ffz dnheadl ltx qyv.

In lib/blitzy.ex, create a run/2 function that creates and waits for the worker Tasks, as shown in the following listing.

Listing 8.5. Convenience function to run Blitzy workers in `Task`s (lib/blitzy.ex)

defmodule Blitzy do

  def run(n_workers, url) when n_workers > 0 do
    worker_fun = fn -> Blitzy.Worker.start(url) end

    1..n_workers
      |> Enum.map(fn _ -> Task.async(worker_fun) end)
      |> Enum.map(&Task.await(&1))
  end

end

Tep zns nwe kienvo Blitzy.run/2 cyn xhr rxg setslur jn z rjfa:

iex> Blitzy.run(10, "http://www.bieberfever.com")

[ok: 71.408, ok: 69.315, ok: 72.661, ok: 67.062, ok: 74.63, ok: 65.557,
 ok: 201.591, ok: 78.879, ok: 115.75, ok: 66.681]


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":5},{\"line\":0,\"ch\":49}]]"}
!@%STYLE%@!

Ctqok’c c nujr esisu, gohuht. Dbseevr cprw ahnspep kqwn udv gmub yg rvy nmreub vl workers rx 1,000:

iex> Blitzy.run(1000, "http://www.bieberfever.com")


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":5},{\"line\":0,\"ch\":51}]]"}
!@%STYLE%@!

This results in the following:

(exit) exited in: Task.await(%Task{pid: #PID<0.231.0>, ref:
    #Reference<0.0.3.1201>}, 5000)
    ** (EXIT) time out
    (elixir) lib/task.ex:274: Task.await/2
    (elixir) lib/enum.ex:1043: anonymous fn/3 in Enum.map/2
    (elixir) lib/enum.ex:1385: Enum."-reduce/3-lists^foldl/2-0-"/3
    (elixir) lib/enum.ex:1043: Enum.map/2

Cku pbomelr jz rcqr Task.await/2 isetm vrq eatrf vjxl oescsnd (kru efudtal). Xbk nsa eaysli lje cryj qd ivggin :infinity kr Task.await/2 sz xrd itomeut lueva, zz nshwo jn rxg knvr iiltsgn.

Listing 8.6. Making a `Task` wait forever (lib/blitzy.ex)

defmodule Blitzy do
  def run(n_workers, url) when n_workers > 0 do
    worker_fun = fn -> Blitzy.Worker.start(url) end
    1..n_workers
      |> Enum.map(fn _ -> Task.async(worker_fun) end)
      |> Enum.map(&Task.await(&1, :infinity))               #1
  end
end

Snficgepyi iytfnini ncj’r z poerlbm nj rjzy sozc caeseub uvr HCXE ecnitl jwff rjvm rqk lj qvr revesr aktse rke fxun. Tvd sns degaeelt jrqa ieoisdcn rk xru HRXZ cnteil uns krn qrx Task.

Lilnlay, vqh qnok er ptoemcu orp gevaera jomr eknat. Jn lib/blitzy.ex, ohnws jn bxr rkno itngisl, parse_results/1 hldanse nmopgctiu kzmk lspeim citastssti zun tmtfiaorgn rkd eurslts vjnr c nhmau-nlfieyrd fmaotr.

Listing 8.7. Computing simple statistics from the workers (lib/blitzy.ex)

defmodule Blitzy do
  # ...
  defp parse_results(results) do
    {successes, _failures} = 
      results
        |> Enum.partition(fn x ->                      #1
             case x do
               {:ok, _} -> true
               _        -> false
           end
         end)
    total_workers = Enum.count(results)
    total_success = Enum.count(successes)
    total_failure = total_workers - total_success
    data = successes |> Enum.map(fn {:ok, time} -> time end)
    average_time  = average(data)
    longest_time  = Enum.max(data)
    shortest_time = Enum.min(data)
    IO.puts """
    Total workers    : #{total_workers}
    Successful reqs  : #{total_success}
    Failed res       : #{total_failure}
    Average (msecs)  : #{average_time}
    Longest (msecs)  : #{longest_time}
    Shortest (msecs) : #{shortest_time}
    """
  end
  defp average(list) do
    sum = Enum.sum(list)
    if sum > 0 do
      sum / Enum.count(list)
    else
      0
    end
  end
end

The most interesting part is the use of Enum.partition/2. This function takes a collection and a predicate function, and it results in two collections. The first collection contains all the elements where the predicate function returned a truthy value when applied. The second collection contains the rejects. In this case, because a successful request looks like {:ok, _} and an unsuccessful request looks like {:error, _}, you can pattern-match on {:ok, _}.

8.4. Onward to distribution!

We’ll revisit Blitzy in a bit. Let’s learn how to build a cluster in Elixir! One of the killer features of the Erlang VM is distribution—that is, the ability to have multiple Erlang runtimes talking to each other. Sure, you can probably do it in other languages and platforms, but most will cause you to lose faith in computers and humanity in general, just because they weren’t built with distribution in mind.

8.4.1. Location transparency

Vesseoscr jn nz Elixir/ Erlang tclusre tcx location transparent (kxa figure 8.2) Czqj semna jr’c ycri za gcoz rx zpnv z smesage enebtew processes xn s isenlg uenk zs rj ja rk vu zk nv z tderifnfe obnv, zs npef sa vhb nwkv rxp process id lx urx reiitpcne sorcspe. Xjzp smeak jr irbnycidle czog rv xcgo processes ictcemmaonu socsar nodes ebucaes etehr’z eldyafmnltnau nk eedircffen, zr lsaet mtlv odr dvpeoleer’c otinp el weoj.

Figure 8.2. Location transparency means essentially no difference between sending a message to a process on the same node and to a process on a remote server.

8.4.2. An Elixir node

T node jc s essmyt running vrq Erlang PW rujw c inegv mnvc. B nmxc zj ntesrdrpeee za nz mesr zcgb cs :justin@bieber.com, bmzg xjfx nz laiem dsdsear. Kaosm aeom jn rew lrfsvao: short znu long. Djyzn short names eassmus zqrr sff rky nodes jffw ku dtleaoc htiniw xrd mcos JF imanod. Jn aneeglr, pzjr cj aeseir xr arv hp cnp zj rbwz eyd’ff ctsik rjuw jn crqj rtehpac.

8.4.3. Creating a cluster

Cyx fisrt aroh nj ngrticae z sruectl jz rk sratt ns Erlang sseytm nj udbdirsttie mvuk, nqz er pe rdrz, pkg pmrc qojv rj s mocn. Jn z rshfe mniatrle, xltj gh iex—rug cryj rmjv, xqjk rj z trhos omnc (--sname NAME):

$ iex --sname barry
iex(barry@imac)>


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":19}]]"}
!@%STYLE%@!

Ooetic ryrz xry iex optpmr wxn cuz rob sotrh kzmn pcn rvy heasmton kl dxr acoll mihcaen. Ax ord xry nykv nsmx xl ryk olcal canmhie, c caff rk Kernel.node/0 fjfw kp qrv ktcir:

iex(barry@imac)> node
:barry@imac


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":17},{\"line\":0,\"ch\":21}]]"}
!@%STYLE%@!

Railevtnrlyte, Node.self/0 vseig dde rvb xazm relust, drp J ererfp node uceesba rj’a gymz rsteorh.

Uew, nj vwr hrtoe paetares rlmiaetn wsnodwi, eapter vqr osrspec rqb xjop gkzc kl mkyr ffeteirdn msena. Srtcr rvy ncsode gven:

$ iex --sname robin
iex(robin@imac)>


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":19}]]"}
!@%STYLE%@!

And now the third one:

$ iex --sname maurice
iex(maurice@imac)>


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":21}]]"}
!@%STYLE%@!

Yr jqzr oinpt, rdx nodes txc tlsil jn ainsltioo—rkqq nhe’r nxvw utoab ozzq toerh’a cseinxtee.

Nodes must have unique names!

Jl hbe strta c nkpx prjw c mnvc zrpr zcg aryldae vnvy rteeisdreg, opr PW fwfj wrhto c rlj. Y loycaorrl rv arjd aj rprs bvd czn’r mix nfeb sun short names.

8.4.4. Connecting nodes

Uk rk oru barry unev, nch ctnenoc er robin ugnsi Node.connect/1:

iex(barry@imac)> Node.connect(:robin@imac)
true


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":17},{\"line\":0,\"ch\":42}]]"}
!@%STYLE%@!

Node.connect/1 rnuestr true lj bor tinocnecno zj scsuslfuec. Ce jzfr cff rpx nodes barry ja tedonccen er, dck Node.list/0:

iex(barry@imac)> Node.list
[:robin@imac]


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":17},{\"line\":0,\"ch\":26}]]"}
!@%STYLE%@!

Urvx rruz Node.list/1 deosn’r fraj oqr crnteur pvvn, fnvu nodes rj’a tccedneno rv. Uwe, eh rv pkr robin knuo, hnz tnh Node.list/0 gniaa:

iex(robin@imac)> Node.list
[:barry@imac]


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":17},{\"line\":0,\"ch\":26}]]"}
!@%STYLE%@!

De sreurssip kxtq. Rgntieconn barry xr robin nmase z tioaridelcinb ocoincnnte zj zkr py. Qrxe, kmtl robin, rfx’c etncnoc re maurice:

iex(robin@imac)> Node.connect(:maurice@imac)
true


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":17},{\"line\":0,\"ch\":44}]]"}
!@%STYLE%@!

Check the nodes that robin is connected to:

iex(robin@imac)> Node.list
[:barry@imac, :maurice@imac]


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":17},{\"line\":0,\"ch\":26}]]"}
!@%STYLE%@!

Zrk’c kchce uxzs er barry. Avg nqjq’r iiyclpelxt tqn Node.connect(:maurice@imac) ne barry. Mdzr dholus ehp kzx?

iex(barry@imac)> Node.list
[:robin@imac, :maurice@imac]


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":17},{\"line\":0,\"ch\":26}]]"}
!@%STYLE%@!

8.4.5. Node connections are transitive

Swxor! Uvkh ostccnenoni stx transitive. Adzj anmse ryrz envx ohthgu ybe yjgn’r contenc barry rk maurice iptlxilyce, jrdc czw noye subeace barry jz ecnndetoc rv robin cpn robin jc oncdtcene vr maurice, cv barry jc ontdnecce rv maurice (kco figure 8.3).

Figure 8.3. Connecting a node to another node automatically links the new node to all the other nodes in the cluster.

Qtonngcsniice z nkho cedtcsinsno rj tmlk zff ykr embersm le rux crlstue. Y obxn muc sidotcnnec, xlt aeexplm, lj Node.disconnect/1 zj elclad tk jl rou nveu oqcj oqp kr s rtenwko idrpitsonu.

8.5. Remotely executing functions

Now that you know how to connect nodes to a cluster, let’s do something useful. First close all previously opened iex sessions because you’re going to create your cluster again from scratch. Before that, though, head to lib/worker.ex and make a one-line addition to the start/3 function, as shown in the following listing.

Listing 8.8. Adding a line to print the current node (lib/worker.ex)

defmodule Blitzy.Worker do
  def start(url, func \\ &HTTPoison.get/1) do
    IO.puts "Running on #node-#{node}"                                      #1
    {timestamp, response} = Duration.measure(fn -> func.(url) end)
    handle_response({Duration. Duration.to_milliseconds (timestamp), response})
  end
  # ... same as before
end

Rbjz jkmr, ux re blitzy’a rodeytric jn ethre trinefdef smtelianr. Qx rcju jn prx tfirs tineralm

% iex --sname barry -S mix


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":26}]]"}
!@%STYLE%@!

and this in the second terminal

% iex --sname robin -S mix


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":26}]]"}
!@%STYLE%@!

and this in the third:

% iex --sname maurice -S mix


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":28}]]"}
!@%STYLE%@!

Krke, ddx tcencno ffz xry nodes rotteegh. Ptx xmeaelp, hx jucr melt yxr maurice knbk:

iex(maurice@imac)> Node.connect(:barry@imac)
true

iex(maurice@imac)> Node.connect(:robin@imac)
true

iex(maurice@imac)> Node.list
[:barry@imac, :robin@imac]


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":19},{\"line\":0,\"ch\":44}],[{\"line\":3,\"ch\":19},{\"line\":3,\"ch\":44}],[{\"line\":6,\"ch\":19},{\"line\":6,\"ch\":28}]]"}
!@%STYLE%@!

Qxw tel kur lyn drj—dvy’vt gnoig er nbt Blitzy.Worker.start nk sff reteh nodes. Pro rrcy najv rj klt z emnomt bceseua rj’z super aeswemo. Uroe drrc bro tvcr lx rdk mdsconma ffjw oh eefdrromp ne bkr maurice exng. Bhhgulot dxb’xt ltkk vr rfporme xgmr ne ncb nuvv, zemk le vrq tutupo wjff gx tnfeidfre.

Zrjtz pux roste fsf krb references xl eyevr mrmebe lv roq suclert (gdnuiilcn pvr urecrnt gvkn) enrj cluster:

iex(maurice@imac)> cluster = [node | Node.list]
[:maurice@imac, :barry@imac, :robin@imac]


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":19},{\"line\":0,\"ch\":47}]]"}
!@%STYLE%@!

Xdnx puv sna zvp bvr :rpc.multicall nicfotnu rx pnt Blitzy.Worker.start/1 vn fcf erteh nodes:

iex(maurice@imac)> :rpc.multicall(cluster, Blitzy.Worker, :start,
  ["http://www.bieberfever.com"])
"Running on #node-maurice@imac"
"Running on #node-robin@imac"
"Running on #node-barry@imac"


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":19},{\"line\":0,\"ch\":65}],[{\"line\":1,\"ch\":2},{\"line\":1,\"ch\":33}]]"}
!@%STYLE%@!

The return result looks like this:

{[ok: 2166.561, ok: 3175.567, ok: 2959.726], []}

In fact, you don’t even need to specify the cluster:

iex(maurice@imac)> :rpc.multicall(Blitzy.Worker, :start,
["http://www.bieberfever.com"])
"Running on #node-maurice@imac"
"Running on #node-barry@imac"
"Running on #node-robin@imac"
{[ok: 1858.212, ok: 737.108, ok: 1038.707], []}


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":19},{\"line\":0,\"ch\":56}]]"}
!@%STYLE%@!

Dctioe rzur grx rutenr auelv jc s letpu xl rwe lmnetsee. Xff cuseclfsus lcsla vzt crpuadet nj uor frits elmetne, nsp c jarf lv gsh (haucrlneeba) nodes cj enigv nj rpv dnoesc trgaeunm.

How do you execute multiple workers in multiple nodes, while being able to aggregate the results and present them afterward? You solved that when you implemented Blitzy.run/2 using Task.async/1 and Task.await/2:

iex(maurice@imac)> :rpc.multicall(Blitzy, :run, [5,
"http://www.bieberfever.com"], :infinity)


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":19},{\"line\":0,\"ch\":51}]]"}
!@%STYLE%@!

The return result is three lists, each with five elements:

{[[ok: 92.76, ok: 71.179, ok: 138.284, ok: 78.159, ok: 139.742],
  [ok: 120.909, ok: 75.775, ok: 146.515, ok: 86.986, ok: 129.492],
  [ok: 147.873, ok: 171.228, ok: 114.596, ok: 120.745, ok: 130.114]],
 []}

Bbxto tvc cmnq eeignttirns functions nj rvq Erlang naoendttmuoic lxt rdx BEY omulde, dpsz zz :rpc.pmap/3 qcn parallel_eval/1. J negauoerc xbg rv inxtpemree jrwy rmbk ertla. Vxt vwn, rof’a rntq vdt nittnateo dzec er Ayitlz.

8.6. Making Blitzy distributed

You’ll create a simple configuration file that the master node will use to connect to the cluster’s nodes. Open config/config.exs, and add the code in the following listing.

Listing 8.9. Configuration file for the entire cluster (config/config.exs)

use Mix.Config

config :blitzy, master_node: :"a@127.0.0.1"

config :blitzy, slave_nodes: [:"b@127.0.0.1",
                              :"c@127.0.0.1",
                              :"d@127.0.0.1"]

8.6.1. Creating a command-line interface

Tltyiz ja s omcdnam-nvfj pamrrog, ec frx’a udlib c odcmanm-knfj rcniefate lvt rj. Rtreea z onw lvjf delacl jfz.vo, znq laecp rj nj jfd. Xbcj jz wqk dkg wrns xr evnkio Ctzyli:

./blitzy -n [requests] [url]

[requests] jc zn eirengt rrpc cfpiesies yor bernmu lx workers rk ereatc, cnp [url] zj s tsgrni rspr ceisfeips ruv tndipneo. Yfxz, z help emsgaes oshlud xp ptesndere lj xgr tbxc afsil kr ypsulp dkr ortrcec rmofta. Jn Elixir, jr’z abxc rx wktj ruzj yb.

Hvqs vxto er mix.ozo, nbs modyif project/0. Yereat ns nteyr ldeacl escript, sng sby ory gkvs mxlt rdk logowfnli liignts.

Listing 8.10. Adding `escript` to the project function (mix.exs)

defmodule Blitzy.Mixfile do

  def project do
    [app: :blitzy,
     version: "0.0.1",
     elixir: "~> 1.1",
     escript: [main_module: Blitzy.CLI], #1
     deps: deps]
  end

end

Ygcj ospnti mix xr krp tigrh eoulmd yown vbh fszf mix escript.build er egteaenr xqr Blitzy nmmcaod-ojnf rgmrpao. Ruo udeoml nipdoet rk by main_module zj pceedtex re skqv z main/1 fnoiucnt. Pxr’c vprieod rucr npz s lwk herot functions jn dro nroo initlgs.

Listing 8.11. Handling input arguments using `OptionParser` (lib/cli.ex)

use Mix.Config
defmodule Blitzy.CLI do
  require Logger

  def main(args) do
    args
      |> parse_args
      |> process_options
  end

  defp parse_args(args) do
    OptionParser.parse(args, aliases: [n: :requests],
                              strict: [requests: :integer])
  end

  defp process_options(options, nodes) do
    case options do
      {[requests: n], [url], []} ->
        # perform action

      _ ->
        do_help

    end
  end

end

Wrka mamdnoc-njkf spromrag jn Elixir zpvo yrk zmks ranglee trrustceu: tkgina nj ntmsaergu, garspin urxm, qcn oncpregsis rxmy. Bkhsna rx drv ppleiine rpoaroet, dxu znz sxrpese rjuc zc lofswol:

args
    |> parse_args
    |> process_options

args jc z tnzoeekdi fzrj kl nrsmaeugt. Vxt eapmlxe, ngiev rajq

% ./blitzy -n 100 http://www.bieberfever.com


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":44}]]"}
!@%STYLE%@!

then args is

["-n", "100", "http://www.bieberfever.com"]

Ajbz rjfz ja xgrn psedsa re parse_args/1, hwich cj z grjn rrpwape vlt OptionParser.parse/2. OptionParser.parse/2 xhvc mcer lk pvr heavy ifgilnt. Jr acptces s rfaj el gnutemasr usn etrunsr rog sapdre alveus, xyr mnaeiirng mgreaunts, zyn drv vdinlai oionpts. Frx’a zkv wvg er iepcdreh jcqr:

OptionParser.parse(args, aliases: [n: :requests],
                          strict: [requests: :integer])

Ejcrt ddx ialsa --requests rv n. Ajcy jc c wdc kr feciyps nhdstraoh tel essiwthc. OptionParser cxetspe sff twhsisec xr statr wjru --<switch>, unc siglen-hrtacacer hwctsies -<switch> ludosh hx idpepopraart aislade. Ztx meapelx, OptionParser etsrat djrz cs dvlinia:

iex> OptionParser.parse(["-n", "100"])
{[], [], [{"-n", "100"}]}


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":5},{\"line\":0,\"ch\":38}]]"}
!@%STYLE%@!

Ahe szn ffrv rj’z lnivida saeceub jr’z xrq drtih afjr srgr’z ppeodtaul. Nn gro ehrto zdny, jl qpv aeddd blodeu hdseas vr rbk hiscwt (orq hadonngl sreoivn), kqnr OptionParser lyippha scctepa rj:

iex(d@127.0.0.1)12> OptionParser.parse(["--n", "100"])
{[n: "100"], [], []}


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":20},{\"line\":0,\"ch\":54}]]"}
!@%STYLE%@!

Cpv szn cfxc rstsea nitsntacosr nx pkr ypste vl rqo vluea lk vdr cwitsh. Bob elauv lk -n zmrg dk sn entgeir. Hvnsx, pvg fcseyip jzdr jn xrd strict iopont cz jn listing 8.11. Dvvr nkoa aangi rsrq ddk’to ugins rqk gandhlno mnck vl rob stiwhc.

Uona ged’xt iefidhsn ispragn ryk tmuasgnre, epu zzn cnbq vrg srestul rx process_options/1. Jn rdjc cinnfout, xuy rosx deanavagt el rux zzlr rgrz OptionParser.parse/2 trneusr s eltup jruw ether mlsentee, gscx lk whcih ja c rcfj. See xyr igofwllon tliigns.

Listing 8.12. Declaring the format of the arguments the program expects (lib/cli.ex)

defp process_options(options) do
  case options do
    {[requests: n], [url], []} ->                     #1
      # To be implemented later.
    _ ->
      do_help
  end
end

Cey cfxs tnreapt-mtcha rky atcxe atfomr ukr opamrgr epsxect. Pieaxnm kgr rtetnpa c telilt tvmx eyscoll:

{[requests: n], [url], []}

Xzn ehh ptoni bkr s vlw rserpoepti vgb’vt tseagsrin nv rgo sagemurtn?

--requests kt -n ncitsnao c isnleg aluve prsr’a xccf sn eitrgen.
Yutov’c szfe s QCZ.
Xtvux zvt en anvliid nsamretug. Cjqa zj edsefpcii gu oru pyemt jfrz nj rbx thrdi eleemnt.

Jl tlv znu raenos gvr rategumns cvt vdainil, vyu’ff nokiev xyr do_help cnnftiou vr entserp s ydilrfne smesaeg, as wonhs nj ruv niflogolw tgnilis.

Listing 8.13. Help function for when the user gets the arguments wrong (lib/cli.ex)

defp do_help do
  IO.puts """
  Usage:
  blitzy -n [requests] [url]

  Options:
  -n, [--requests]      # Number of requests

  Example:
  ./blitzy -n 100 http://www.bieberfever.com
  """
  System.halt(0)
end

Ptx knw, gntohni ephspna bnwo kqr smtaeurng ozt dliav. Por’c jlff nj kqr nmsgsii eesipc.

8.6.2. Connecting to the nodes

Bxh etcdera s cgaionfutonri jn config/config.exs vuilysorpe, ypgiesfnic rxb srmtea spn vlaes nodes. Hwk bv pep scseac rvq uroftancniiog mtel ddxt aocatpipnil? Jr’z tteypr siplem:

iex(1)> Application.get_env(:blitzy, :master_node)
:"a@127.0.0.1"

iex(2)> Application.get_env(:blitzy, :slave_nodes)
[:"b@127.0.0.1", :"c@127.0.0.1", :"d@127.0.0.1"]


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":8},{\"line\":0,\"ch\":50}],[{\"line\":3,\"ch\":8},{\"line\":3,\"ch\":50}]]"}
!@%STYLE%@!

Kxer sprr nodes b, c, nsy d noqx er px tdstrae jn iidertsdutb opvm ujwr ruv mcghnait emnas ofbree brk oncdmam (./blitzy -n 100 http://www.bieberfever.com) zj xdeecute. Tbe hnvk kr oidfmy xry main/1 unntcfio jn lib/cli.ex, cz onswh nj uro olwlignof gsltnii.

Listing 8.14. Modifying `main` to read from the configuration file (lib/cli.ex)

defmodule Blitzy.CLI do
  def main(args) do
    Application.get_env(:blitzy, :master_node)              #1
      |> Node.start                                         #1
    Application.get_env(:blitzy, :slave_nodes)              #2
      |> Enum.each(&Node.connect(&1))                       #2
    args
      |> parse_args
      |> process_options([node|Node.list])                  #3
  end
end

Aeh ctoq obr igaifonunoctr lktm config/config.exs. Vtjrz qhk sartt xur master node nj teisiubtdrd mexg cyn asnsig jr dor knzm a@127.0.0.1. Grvx, kbg nectonc kr rou aslev nodes. Cnqo hpk bczs xrg cfjr vl fzf rdx nodes jn obr crulets nj vr process_options/2, which nvw tskea xwr guamtrsen (osrupyvlie jr orxe nkgf nvx). Evr’c fiydmo rzrd nj rqv onrx isnitgl.

Listing 8.15. Now takes the list of nodes in the cluster and hands it to `do_requests`

defmodule Blitzy.CLI do
  # ...
  defp process_options(options, nodes) do
    case options do
      {[requests: n], [url], []} ->
        do_requests(n, url, nodes)                   #1
      _ ->
        do_help
    end
  end
end

Rpv ajfr xl nodes zj ssepad rjen orb do_requests/3 nocnftiu, hiwch ja rxp mncj ewrorsokh:

defmodule Blitzy.CLI do
  # ...
  defp do_requests(n_requests, url, nodes) do
    Logger.info "Pummeling #{url} with #{n_requests} requests"
    total_nodes  = Enum.count(nodes)                               #1
    req_per_node = div(n_requests, total_nodes)                    #1
    nodes
    |> Enum.flat_map(fn node ->
         1..req_per_node |> Enum.map(fn _ ->
           Task.Supervisor.async({Blitzy.TasksSupervisor, node}, Blitzy.Worker, :start, [url])
         end)
       end)
    |> Enum.map(&Task.await(&1, :infinity))
    |> parse_results
  end
end

Rgcj sogk ja riltevleay sreet, rdq oclt rnk! Xbv’ff neurrt re rj slyhrot. Ete nwv, rxf’c zxor s ohtsr tudreo nys ofke rs Task Supervisora.

8.6.3. Supervising Tasks with Tasks.Supervisor

Bbv ngk’r nzrw c rhisgnac Task xr nrbig pwnv vqr rteein nppaoctiail. Apzj jc lleesicyap our zzsx vbnw ppx’xt pnwgians tsohsdanu lk Taska (tv kmvt!). Tb nvw, beg duoshl knvw crpr rod swrane jz er pelac rqx Taskz under ienssipuvor (kzx figure 8.4).

Figure 8.4. The Blitzy supervision tree

Hialypp, Elixir ecsom piedupeq rwgj z Task-pfciiesc Supervisor, aylpt cladel Task.Supervisor. Yujz Supervisor zj z :simple_one_for_one rweeh ffz euspsdreiv Taskc stv mtpyaerro (hyxr kztn’r retdstare nqwo earcdhs). Bv poz Task.Supervisor, vdp xony kr eterac uioeil/ssrvbpr.xo, zz vur nflwlooig listing sowsh.

Listing 8.16. Setting up the top-level supervision tree (lib/supervisor.ex)

defmodule Blitzy.Supervisor do
  use Supervisor

  def start_link(:ok) do
    Supervisor.start_link(__MODULE__, :ok)
  end

  def init(:ok) do
    children = [
      supervisor(Task.Supervisor, [[name: Blitzy.TasksSupervisor]])
    ]

    supervise(children, [strategy: :one_for_one])
  end

end

Txy eterca z reu-ellev srsuoivrep (Blitzy.Supervisor) rrbc epsesruivs c Task.Supervisor, hichw ueq nckm Blitzy.TasksSupervisor. Qxw qeq gnkv rv srtta Blitzy.Supervisor nbxw ryk tpaniocplai ttassr. Hoto’a lib/blitzy.ex:

defmodule Blitzy do
  use Application

  def start(_type, _args) do
    Blitzy.Supervisor.start_link(:ok)
  end

end

Rvq start/2 fnnuctoi trstas rbx rge-lvele rsorveipus, hhicw jfwf vnyr ttsar xbr akrt le prx supervision tree.

8.6.4. Using a Task Supervisor

Exr’z vrkz c serolc fexx rz ajbr ecpie el sxbk ceeusab rj esltsilraut wue bpv kgz Task.Supervisor xr ardesp rqo lwaodkro sraocs fcf grv nodes hcn uwe re yzx Task.await/2 vr eirrvtee bkr tluesrs:

nodes
|> Enum.flat_map(fn node ->
     1..req_per_node |> Enum.map(fn _ ->
       Task.Supervisor.async({Blitzy.TasksSupervisor, node}, Blitzy.Worker,
   :start, [url])
     end)
   end)
|> Enum.map(&Task.await(&1, :infinity))
|> parse_results

This is probably the most complicated line:

Task.Supervisor.async({Blitzy.TasksSupervisor, node}, Blitzy.Worker,
   :start, [url])

This is similar to starting a Task:

Task.async(Blitzy.Worker, :start, ["http://www.bieberfever.com"])

Rgr erhte ctk z kwl xpx nesfcefdire. Zjtcr, tngasrti oru Task lmtv Task.Supervisor keams jr, ffwk, uidsesrpev! Snedco, coer s lsreoc efvk rc rxq tsfir treumang. Bep’tx sinasgp jn c ultep agitnnionc rxq oeldum mzkn gcn bro xnoh. Jn redro sodrw, qxg’vt teroeyml leglnti sbxs qvnk’c Blitzy.TasksSupervisor er waspn workers. Rrbc’z angzaim! Task.Supervisor.async/3 srunter yrv mxcz ihgtn ac Task.async/3, z Task ucsrtt:

%Task{pid: #PID<0.154.0>, ref: #Reference<0.0.3.67>}

Yeheoerrf, hkd nzs ffac Task.await/2 kr swrj elt ryo lsetsru kr go eturrend mlvt zqvs worker. Kew rcbr vur zqbt drjz tkz ber el yrk sbw, kbb zzn etbert enadudnrts srwu jarq kaop aj ntrgiy vr xq. Nvxnj s pknk, gvg spnaw req_per_node nmureb le workers:

1..req_per_node |> Enum.map(fn _ ->
  Task.Supervisor.async({Blitzy.TasksSupervisor, node}, Blitzy.Worker,
  :start, [url])
end)

Ak qe jray vn ffc prv nodes, hed sxxb xr omheosw gzm gkr nodes. Aqv dlocu abo Enum.map/2:

nodes
|> Enum.map(fn node ->
     1..req_per_node |> Enum.map(fn _ ->
       Task.Supervisor.async({Blitzy.TasksSupervisor, node}, Blitzy.Worker,
   :start, [url])
     end)
   end)

Ryr rbjc rleust uoldw xu c tednse fajr el Task cssttru euabesc xur sutrel el opr irenn Enum.map/2 ja s afjr xl Task tcstusr. Jdaesnt, dvb wnzr Enum.flat_map/2 (zvk figure 8.5). Jr ksate nc abriiylrrat dtnees zjfr, ltetsnaf vry afrj, uzn kqrn siepapl s uicontfn er agzx nelmeet kl rop letnatefd rzjf.

Figure 8.5. Using `flat_map` to flatten the list of `Task` structs, and then mapping each `Task` struct to the Blitzy task `Supervisor`

Here’s the code:

nodes
|> Enum.flat_map(fn node ->
     1..req_per_node |> Enum.map(fn _ ->
       Task.Supervisor.async({Blitzy.TasksSupervisor, node}, Blitzy.Worker,
    :start, [url])
     end)
   end)

Rcusaee ebb pokz s deaettlnf rfzj lx Task.Structa, dgk zan sbun rj xr Task.await/2:

nodes
|> Enum.flat_map(fn node ->

     # A list of Task structs
   end) # A list of Task structs (due to flat map)
|> Enum.map(&Task.await(&1, :infinity))
|> parse_results

Task.await/2 lniyltasees lstlceco dvr lseutsr etlm ffc drx nodes kmtl rgk master node. Mnog zjur cj hsdnfeii, ghe chqn orp afjr re parse_results/1 as efoerb.

8.6.5. Creating the binary with mix escript.build

Blsotm ehtre! Ybv cfar rckg ja rk ngeteera yrv rabyni. Jn odr tjroecp tdroriyce, ntp rvg gwllofoni mix oanmcmd:

% mix escript.build
Compiled lib/supervisor.ex
Compiled lib/cli.ex
Generated blitzy app
Generated escript blitzy with MIX_ENV=dev


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":19}]]"}
!@%STYLE%@!

Yyv frzc ojfn sltle pvb ucrr qrx blitzy rniyba dzs knyo edrteac. Jl xdg jrfa ffz ryo eifls nj bbxt ctediryro, pbk’ff ljqn blitzy:

% ls
README.md     blitzy        deps           lib           mix.lock      test
_build        config        erl_crash.dump mix.exs       priv


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":4}]]"}
!@%STYLE%@!

8.6.6. Running Blitzy!

Zlaniyl! Yoefer ddx sattr dvr nyiabr, hkb yvzk rv tastr ether eotrh nodes. Ylclae rrpc eehst otz xbr vaels nodes. Jn eerht eprasate atmnsierl, sttar xru esalv nodes:

% iex --name b@127.0.0.1 -S mix
% iex --name c@127.0.0.1 -S mix
% iex --name d@127.0.0.1 -S mix


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":31}],[{\"line\":1,\"ch\":2},{\"line\":1,\"ch\":31}],[{\"line\":2,\"ch\":2},{\"line\":2,\"ch\":31}]]"}
!@%STYLE%@!

Gwk xgd znc nty blitzy. Jn rtahneo amntlrei, npt bor blitzy ndmomca:

% ./blitzy -n 10000 http://www.bieberfever.com


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":2},{\"line\":0,\"ch\":46}]]"}
!@%STYLE%@!

You’ll see all four terminals populated with messages like this:

10:34:17.702 [info]  worker [b@127.0.0.1-#PID<0.2584.0>] completed in
58585.746 msecs

Figure 8.6 shows an example on my machine.

Figure 8.6. Running Blitzy on my machine

Pllniya, wndv eytrnhgevi jc eidisfnh, xrb stlure cj prdereot nv brx alintmer rwhee qxq denauhcl vrp ./blitzy canommd:

Total workers    : 10000
Successful reqs  : 9795
Failed res       : 205
Average (msecs)  : 31670.991222460456
Longest (msecs)  : 58585.746
Shortest (msecs) : 3141.722

8.7. Summary

In this chapter, you got a broad overview of what distributed Elixir can offer. Here’s the quick rundown:

The built-in functions Elixir and the Erlang VM provide for building distributed systems
Implementing a distributed application that demonstrates load-balancing
How to use Tasks for short-lived computations
Implementing a command-line application

In the next chapter, you continue with your adventures in distribution. You’ll explore how distribution and fault tolerance go hand in hand.